We just got the top community score on ARC-AGI-3's 25 public games: 78.4% and 160/183 levels cleared, where the best frontier LLM session scores 7.8%. No training or demonstrations allowed. Paper: arxiv.org/abs/2607.01531 Blog: david-courtis.github.io/opin… w/ @WenhaoLi29 @ScottSanner #AI #worldmodel #agentic #AGI @arcprize @fchollet

Learn more about our work at our blog, where we’ve deconstructed some empirical results about our system: david-courtis.github.io/opin… And read our full paper here: arxiv.org/abs/2607.01531 w/ @WenhaoLi29 @ScottSanner

Implementing this, OPINE-World reaches 78.4% on the 25 games, with near-human action efficiency on its 20 wins. The specific model we used scores 1.5% when it plays on its own (the current frontier LLM scores 7.8%). The most developed prior method, on comparable models at similar reasoning effort, peaks at 63.7%.

Specifically: 1) An LLM's internal world model cannot be inspected or trusted. We address this by having the agent write the game's dynamics as an object-oriented Python program, accepted when it perfectly models the world or addresses faults as a testable hypothesis. 2) LLMs explore blindly and fall into “hypothesis lock-in”. We address this with ontology error, a Bayesian measure of how much of the game the program still can't explain. The agent spends its moves where that score is highest.

We investigated and found that pure LLMs and existing approaches fail this benchmark for two reasons: their internal model cannot predict the world in an accurate but fault-tolerant manner, and they struggle with hypothesis generation and testing in unknown environments. We propose enhancements to address these shortcomings in our new paper “OPINE-World: Programmatic World Modeling with Ontology-error-Prioritized Interactive Exploration” (arxiv.org/abs/2607.01531)