Shipping to prod in a $100M+ ARR unicorn startup w/ local AI. 4x DGX Spark (GLM 5.3 Flash), 1x M5 Max 128GB (Qwen 3.8 Flash Next), 1x RTX 5090 (Qwen 3.8 27B).

Joined February 2018
That's basically what you're seeing here: M3.1 Flash's road output looks like a 240p downsampled YouTube video (with road antialiasing/dithering, etc.), while DeepSeek's output looks like an upsampled Google Maps navigation screen.
1
42
Your prompt is incredibly contradictory in multiple places, but this line right here means that MiniMax absolutely wins the benchmark according to the stated primary criteria. You explicitly state that it should "not be like a video game", but you imply throughout that it should be an exaggerated cinematic style (e.g. "like a video game"). But despite your mixed messages, explicit instructions should always override conflicting implicit undertones. So again, MiniMax did a better job with instruction following on this adversarial prompt.
10
GDPVal's examples are all basically "hey, you have this specific task, and all of the external information you should need is included in this 1 attached file you can read once and are never expected to mutate." Unsurprising this benchmark is generation-heavy.
1
16
I am so tired of Pagoda benchmarks. Here's my Pagoda benchmark for DS 4.1 Flash 1017.
38
Replying to @migtissera
Forgive me for the heuristic, but I'm pretty skeptical of this. Every finetune of Qwen always says it's unequivocally better, and the claims here are VERY similar to the claims Qwopus makes (same quality, shorter responses). My Qwopus tests disproved their claims and I don't know why I should have faith that the same wouldn't apply here. Unless you're trying to suggest that you just generally do a better job than the entire Qwen team, it's hard to believe that you made a pure upgrade to their model with no tradeoffs whatsoever. I can, however, believe that it's better for the way you use it and maybe the way some others expect to use it.
2
319
Comparing speed and outcomes between GLM 5.3 Flash and DeepSeek 4.1 Flash 1017 across 4x DGX Sparks.
1
59
Replying to @andrewho03
Lol. Just call it ragebait and pretend you knew all along. You have enough evidence to make it believable. Use it to your advantage.
86
Replying to @teortaxesTex
The reason it throws people off is that it has "Flash" in the name, which (reasonably) implies "fast responses". They should have instead named it after size/efficiency rather than picking a name that implies speed. Chinese companies aren't good at English product names, e.g.:
2