Last weekend I had an idea: a multiplayer world where you build anything just by describing it I invited a few friends to play, and we lost hours to it This weekend, you and your friends can get lost in Allcraft too!

Game development remains one of the most-requested, and most-challenging, categories on Arena. @iamwaynechi, PhD candidate at Carnegie Mellon University and research intern at Arena, just walked us through GameDevBench: a benchmark built from real tutorials that turns game development into verifiable, deterministic tasks. How do top frontier models perform on tasks a human beginner could complete in under an hour, and is the biggest bottleneck coding or multimodal understanding? Check out Wayne’s full talk to hear results and learn more about GameDevBench: youtube.com/watch?v=yoHcvrmE…

Week 6 of building a fishing game entirely with AI What started as an experiment is now a playable game you can actually download. The creator has zero game dev experience. Claude handles all the coding in Godot, while Claude and now GPT-6 Astra help create the 3D assets. Characters are made with Tripo3D. This week added new islands, docking areas, fishing progression, improved lighting, redesigned buildings and an almost finished starting zone. No one-shot prompting. Just 6 weeks of building, testing, fixing and seeing how far a fully AI-made game can go.

We're starting to leave the territory where you'd test an LLM by e.g. "create an svg of pelican on a bicycle". As one idea to generalize it, I was interested what Opus 5 would do if I gave it the first paragraph of the Lord of the Rings, a 1M token budget (~$10) and asked for three js render of it. Opus went off for ~2 hours and wrote 5500 lines of code that (procedurally) rendered the story. It's kind of janky but fun. But it's a bit mindboggling that the LLM has to place and orchestrate various polygon assets in (x,y,z) coordinates and write code that animates it all, and that it even does anything at all. I also like this kind of examples because no one in their right mind would ever spend the time to write something this custom but LLMs have all the stamina and patience in the world, so it's an example where we go from "no one would ever do this" to "sure, why not, it's ~free". There might be a lot more. But I'm excited about creating hyper custom worlds that you can imagine dropping players into, e.g. here to participate in the LoTR story as a spectator NPC, or one of the characters, or etc. Something like an ephemeral GTA of X on demand. Last thought is that the domain of worlds/games exposes a weakness in LLMs: they can't easily audit their work because they aren't able to efficiently and natively perceive videos or play games within them. Here, Opus 5 had to very slowly and painstakingly take screenshots at different points, and it messed up a few times and created a bunch of jank. An example of raw capability (multimodal, gameplay) that I think is still quite lacking.

We love our fast and cheap models for computer use!

Today we’re launching AutoEval: a new evaluation methodology that ranks models using reward models based on millions of real Arena user preferences. Highlights: - High-quality evaluation signals calibrated on real preference data - Strong alignment with live human evaluations - Evaluations that are several orders of magnitude faster (hours instead of days) - Support for Text, Vision, Image, and Code Arena AutoEval enables us to evaluate newly launched models much faster and share results with the community sooner. We’ll now show AutoEval estimated scores for new models directly on the leaderboard. More details in the thread. 🧵

Opus 5 test with a first-person shooter prototype, one shot. Took like 1.5 hrs It not only created the entire game but also spawned bots to play in multiplayer. Flight mechanics may be under-tuned, but... It's easily the most powerful model of all time full stop.