Zhiting Hu

MiniMax M3.1 Flash reshapes BuildingBench’s cost–quality frontier! Another open model making moves in 3D 🏗️🔥 🎈 73.3 overall, ranking #12. 🎈 Above SWE-2’s 67.3, taking its place on the Pareto frontier. 🎈 Close to the previous DeepSeek V4.1 Flash’s 72.9—and currently free through OpenRouter! Its best reconstruction hits 88.8 on Melbourne’s Australian Centre for Contemporary Art. How does it look alongside DeepSeek V4.1 Flash 1017 and #1 Opus 5.5? Comparison video below 👇

See DeepSeek’s 3D leap with your own eyes 👀🏗️ DeepSeek V4.1 Flash 1017 🆚 DS V4.1 Flash 🆚 GPT-6 Astra: three buildings, side by side. The new open model improves on 11 of 12 buildings in BuildingBench, jumping from 72.9 → 81.9—closing in on Astra’s 84.3 at less than half the median cost ($3.01 vs $7.05/task). Open models are getting seriously good at building worlds 🔥

Wow🙀🙀 DeepSeek V4.1 Flash 1017 joins BuildingBench’s top five—an exciting leap for open models!! 🔥 We tested its 3D building reconstruction from real photos. An open model delivering this level of spatial reasoning at this cost is exciting to see! 👀 🎈 81.9 overall, up from 72.9 for the previous version. A 9-point jump! 🎈 Slightly higher than Fable 5.1’s 81.4, and essentially level with Sonnet 5.5’s 82.1. 🎈 Estimated median cost: $3.01/task. About 1/11 of Fable 5.1’s cost and 1/15 of Sonnet 5.5’s.

🎉Excited to see two new US open models from this week: 📘Meta’s Muse Spark Open 📗NVIDIA’s Nemotron 3.5 Super VL How well can they build in 3D? 🏗️👀 We tested both on BuildingBench: reconstructing real buildings from photographs. 📘Muse Spark Open (max): 56.9 at an estimated $8.59/task—comparable to Kimi K3’s 54.0. 📗 Nemotron 3.5 Super VL (max): 37.8 at an estimated $0.76/task—among the cheapest tested, with performance near GLM-5.3 Flash’s 33.0. DeepSeek V4.1 Flash remains well ahead of both: 72.9 at an estimated $2.33/task. GLM-5.3 FlashX also scores higher, at 68.0. Neither newcomer reaches the cost–quality frontier. More open models to choose from, and plenty of room to improve spatial reasoning! Both ran in Claude Code at max effort. Costs are estimated median per task.

Watch how GPT‑6 Astra + WorldBox tackle MazeBench, from its first action to gem💎 #61—32,980 live actions and around 1B tokens. Building bridges with boxes, launching across gaps, solving puzzles that span multiple rooms… all while the map fills in. Remember, test, revise, repeat, and grab that gem. WorldBox offers the supporting layer for Astra.

Here’s what that progress looks like across the maze. 🗺️ Rooms visited by the end of each run: 🥉 Opus 5.5: 68 🥈 GPT‑6 Astra: 113 🥇 GPT‑6 Astra + WorldBox: 230 of 256 Brighter tiles mark more frequent visits. With WorldBox, the agent reaches almost 90% of the rooms and steps onto nearly twice as many distinct tiles as the Astra baseline. WorldBox starts with empty memory and no skills. The agent writes its own notes and Python skills, reconstructs room geometry from camera views, and tests candidate moves in the sandbox. Remembering where you’ve been helps make the next exploration decision. What does that reasoning look like inside a difficult room? 👇

🚨New level unlocked! 🚨 We doubled GPT‑6 Astra’s score on MazeBench with WorldBox, our agentic sandbox for spatial reasoning. Astra: 23 gems 💎💎 Astra+WorldBox: 61 gems 💎💎💎💎💎 MazeBench, by @patience_cave, is a tough test of long-horizon spatial reasoning: 256 interconnected rooms where even Opus 5.5 used 2.2B tokens and collected just 13 of 100 gems. WorldBox provides agents with an agentic sandbox, for managing long-term memory, inferring complex mechanics, and simulating reasoning. Watch the sandbox take shape and the agent tackle increasingly tricky puzzles. 🧵

Gemini 4 Argon makes a big leap in 3D reconstruction 🏗️🚀 On BuildingBench, Argon (high) scores 81.2—up from Gemini 3.8 Flash’s 54.8, placing #6 among all 30 models. Near Fable 5.1 quality at ~1/5 the cost. Near Sonnet 5.5 quality at ~1/7. 👀 💸 Estimated median cost: $6.52 per building 🎨 Appearance score: 94.1, the third-highest 🏛️ Best reconstruction: Royal Exhibition Building, scoring 92.1 It doesn’t move the Pareto frontier: GPT‑6.1 Sol still scores higher at lower cost. But this is a substantial step forward for Gemini. Congrats @GeminiApp team!

Two models this week: Haiku 5.5 and Mistral Large 4 take on BuildingBench 🏗️👀 🚀 Haiku 5.5 (max) jumps from its predecessor’s 26.7 to 67.1! It beats Sonnet 5 by 13 points at roughly 1/11 the cost: $1.34 vs. $15.03 per building. 🇫🇷 Mistral Large 4 (max) makes its BuildingBench debut at 52.0, using the Claude Code harness. At $33.21 per building, cost-efficiency has room to improve. Haiku 5.5 narrowly misses the Pareto frontier: SWE‑2 scores almost identically and is still free during its promotion. 👀