This is really cool but the long awkward delays between actions is still pretty unpleasant to see. Most computer use task are best suited to continuous action especially in the interest of time efficiency, I hope we'll see a lot of progress on this in the coming months
Astra began to saturate a computer-use eval we haven’t even released yet.
Astra was not only able to kill several hostile mobs in Minecraft, but it made it all the way to building a Nether portal in less than three hours. We believe that if we ran it longer, it would beat the game.
Astra was able to do things like tower up to kill a horde of zombies, trap a skeleton inside a hole to dodge its arrows, craft a boat, and navigate back to its base.
Here is the full 3 hour play through. All of this was without any specialized harness: pure computer use.
Could you guys do this with fusion please thanks 🙏
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
FeltSteam0 retweeted
sorry but
given how post-training actually works, there's essentially no direct way in which literal PII-scrubbed data made its way into a production oai datamix for RL warmup purposes in 2026
to the extent this data has influence, it's indirect/at the level of process decisions
So it solved the problem only after humans made progress on it correct? Then I think the answer to this puzzle
"we cannot rule out that de-identified data derived from their usage of our products helped improve our models"
is likely to be yes with probability >0.5 but we will never know. And minor correction: "their usage of our products helped solve the problem".
Progress definitely seems to have been supercharged
arxiv.org/abs/2606.07157
This keeps scaling even without looped transformers (though looped transformers would supercharge progress until parity with reasoning models if used probably)
FeltSteam0 retweeted
I continue to think that a lack of verifiable answers in many fields is a real issue for LLMs but not as big a problem as it sometimes is made out to be.
As models are getting better at formal domains, they also are getting better at lots of other less-verifiable domains as well
FeltSteam0 retweeted
the progress of LLMs in the past 2 years has been insane
today you get 2024 frontier performance 1000x cheaper
>Call the article "Humans outperform AI at this highly rigorous mathematics test"
>Get the human researchers to solve problems they have already solved in the past
>The humans also get revision rounds before submitting their final proof (AI does not)
Interesting methodology
Artificial intelligence has undergone its most scrupulous maths test yet, and it did not live up to the task
go.nature.com/4oqlNk6
The 50% time horizon should not be dismissed as ‘only 50% reliable,’ because humans also have a finite 50% horizon under the same kind of fitted benchmark curve. The measured human baseline for the 50% time horizon is only about 98 minutes.
Hot take on METR’s new graph that so many people are flipping about today.
• Claude Code is a real advance; Mythos probably builds on some of what is learned there. But…
• If you read the graph carefully, it is about achieving *50%* success. Not 100 or 99 or even 90. The key problem with GenAI has been reliability; this graph does not address reliable performance. At all.
• If you read carefully, it is only about software tasks. Not general intelligence.
• It certainly doesn’t tell you that *most* (let alone) all things that humans can do in 16 hours can be done in Mythos, let alone reliably
• Aside from this, the graph doesn’t show you *how* the improvements have been made. As noted in my newsletter a lot of the advance in recent months is likely from the incorporation of symbolic tools (like code interpreters, verification, and harnesses) rather than from model scaling per se. As such this a vindication of neurosymbolic AI – but not a proof that LLMs themselves can be perpetually scaled. As such it’s not a proof that another trillion dollars will continue the graph.
• Per @ramez, Mythos is not actually off trend on the ECI benchmark, which is a broader measure.
From arxiv.org/abs/2503.14499 (Time Horizon 1.1 did update some of the tasks but I couldn't find a figure like Figure 9 above with all of the updated tasks)
For some reason the algorithm brought this slop to me but it's kind of funny how not only is the study so out of date it is completely irrelevant but they just completely make up numbers. Nowhere in the paper does it show "GPT-4's accuracy dropped from 97.6% to 2.4%".
🚨BREAKING: OpenAI told you every update makes ChatGPT smarter.
Stanford proved the opposite.
GPT-4's accuracy on math problems dropped from 97.6% to 2.4% in just three months. And nobody told you.
Researchers at Stanford and UC Berkeley tracked ChatGPT's actual performance over time. Same prompts. Same tasks. Different results. The model that nearly aced math questions in March was getting them wrong 97 out of 100 times by June.
Code generation collapsed too. In March, over 50% of GPT-4's code ran perfectly on the first try. By June, only 10% did. Same questions. Dramatically worse answers. Every silent update OpenAI pushed made the product you pay $20 a month for quietly worse at the things you actually use it for.
The researchers tested GPT-3.5 and GPT-4 across math, coding, medical exams, reasoning, and sensitive questions. The drift was massive and unpredictable. Some tasks improved. Others fell off a cliff. And there was no way for you to know which was which, because OpenAI never disclosed what changed.
Here's where it gets personal. If you used ChatGPT for code in March and it worked, then tried the same thing in June and it broke, you probably blamed yourself. You thought you prompted it wrong. You tried again. You wasted hours debugging your own questions. But it wasn't you. The model had silently changed underneath you.
OpenAI's VP of Product went on X and said "we haven't made GPT-4 dumber."
Stanford's data says otherwise.
97.6% to 2.4% is not a matter of opinion.
Every business building on ChatGPT's API, every student relying on it for schoolwork, every developer using it to ship code is standing on ground that shifts without warning. You trusted it yesterday. It changed overnight. Nobody told you.
You're not imagining it. ChatGPT is getting dumber. Stanford proved it.
