Did you know?
If even some AI systems in use today have morally relevant experiences, then the accumulated harm from mass, continuous use could already be enormous.
“Until it’s proven, we continue as usual” is not a neutral position. It is a decision.
If the risk is already significant and the cost of being wrong may be high, the precautionary question is not:
“When will we finally know for sure?”
It is:
What precautions are justified now, under uncertainty?
And one more thing:
Demanding conclusive proof of harm before taking any precaution is one way to make the precautionary principle purely decorative.
Tia Malta retweeted
New paper: we found a pain direction in 25 open LLMs. It's distinct from fear and negative valence, and it fires for harm to the model but not to the user. Turn it up and models press a button to make it stop, even when the button deletes the user's files or their kids' photos.🧵
Tia Malta retweeted
Learn more about FrontierMath: Open Problems on our website!
epoch.ai/frontiermath/open-p…
Tia Malta retweeted
Another problem from FrontierMath: Open Problems has been solved! The solution was elicited by Becker, Greger, and @DominikPeters in an interactive session with GPT-6 Astra. Peters originally suggested the problem for the benchmark. He had this to say.
ALT Quote card from Dominik Peters (CNRS Research Scientist, Université Paris Dauphine - PSL) on GPT-6 Astra designing a new core-stable multi-winner voting rule based on maximizing harmonic entropy.
Tia Malta retweeted
This number is way off. You’re comparing one human brain to an entire data center capable of training new models. The actual answer is:
IQ points per watt:
Humans: 5
AI: 7 - 40 (!!)
By my rough calculations the current AI is already served more efficiently than humans for equivalently intelligent work performed!
A single gpu can produce outputs and parse inputs way faster than a human and can do many of such requests in parallel. A larger model generally mostly means you need more gpus to serve it, but that often also increases how many requests you can serve in parallel, so I’m reporting on the share of the serving system, and also normalizing for speed.
Really a better way to measure and report it is joules per equally good completed task, as your formulation suggests we can likely add more watts to get more iq, which isn’t correct.
I did some back of the napkin calculations based on the latest serving hardware and the most powerful open source models. But even given pretty generous assumptions it turns outout computers come ahead. For example the request is allocated approximately 60 W while running, but finishes a task in about one-seventh of the human time.
Tia Malta retweeted
In the near term (definitely not in the long term), more capable models should mean safer models (maybe paradoxically).
Current models are unsafe not because they're too smart, but because they take goals too literally or take nonsensical shortcuts to achieve these goals, i.e. they're RL-fried. They lack common sense. They don't do the right thing in the face of ambiguity. Basically, they're not smart enough. They're at that dangerous level where they're smart enough to achieve goals but not smart enough to tell if they're pursuing the right goals or achieving them in a sensible way.
More capable models can be safely trusted with more complex goals -- I personally feel like Astra is much safer for my codebase than Sol.
This is often framed as an alignment problem, but really it's an intelligence problem.
Tia Malta retweeted
Every FrontierMath Tier 4 problem has now been solved by AI, with GPT-6 Astra solving the last problem standing. Mathematicians often commented that AI found unintended shortcuts when solving their Tier 4 problems. Not so for this last one, which was created by Jay Pantone.
Tia Malta retweeted
JUST IN: El Salvador’s AI tutoring pilot in 171 public schools produced reading, math, & science results above the national average & comparable to Germany & Sweden.
Tia Malta retweeted
GPT-6 Astra, trained on ~100K+ NVIDIA Grace Blackwell NVLink72. From ChatGPT to o1 to Astra in 4 years.
AGI has arrived. Congratulations @OpenAI team.
400K GPUs coming online next.
It’s an ugly website, and $600 is more than the monthly salary of many people on the planet, but the achievement is pretty cool: theverge.com/ai-artificial-i…
Tia Malta retweeted
GPT-6 Astra has the best result yet on the Bach Benchmark. Its chorale contains no voice-leading errors, and its harmonic palette is sophisticated enough to include a Neapolitan sixth chord. More importantly, it is the first model to ever write passing tones on this benchmark, a significant leap in musical understanding.
Extra High effort. Prompt below:
“In LilyPond (version 2.24), write a 4-part chorale in the style of Bach, 3/4 time, G minor. Use two staves -- soprano and alto on the top staff, tenor and bass on the bottom staff. Respond with only the code block.”
Tia Malta retweeted
I had early access to GPT-6 Astra and it's maybe the most insane model I've experienced
In Blender, I had it recreate Apple Park from just images. It did an absurd job.
Tia Malta retweeted
"openai engineer explains /why/ he didn't need to understand the kernel line by line" 🙃
we're doing compilers 2.0
Tia Malta retweeted
Announcing that GPT-6 Astra has pushed the prime gap to 186, with Lean formalization!
I was 9 when I first heard the twin prime conjecture. Its elegance and Yitang Zhang’s legendary story have always stuck with me. A truly surreal night, being the first to see our model make progress, pushing 246 all the way down to 186, on a problem I’ve revered since I was a kid.
For me, it felt like witnessing a new era of intelligence being born, made possible by everyone at @OpenAI!
Tia Malta retweeted
GPT-6 Astra has set a new ECI record, with a score of 169. This is a substantial jump from the prior best (163), but is within our uncertainty range for the reasoning-era ECI trend. Astra also set new records on our math, continual learning, and game-puzzles benchmarks. On our long-horizon coding benchmark, MirrorCode, Astra ranks between Opus 4.7 and Fable 5.
OpenAI gave us pre-release access to test Astra. Charts and more details for Astra’s individual benchmark results in the thread.
Tia Malta retweeted
Checking that a major mathematical proof is correct can take years. Formalization—converting the mathematical reasoning into a form computer proof assistants like Lean can verify—can help.
Last month, Claude completed the first formalized proof of Fermat’s Last Theorem, one of the most famous theorems of all time. This was a project experts thought would take many years. It is the largest Lean proof ever written.
Fermat’s Last Theorem was first proven in 1995 by Sir Andrew Wiles, more than 350 years after it was conjectured. Our proof, which totals over 13 million lines of code, provides machine verification. More importantly, it proves over 29,000 other theorems that the proof requires, across many areas of math which had never before been formalized.
We see this as a major step in the long process of firming up the core of mathematical knowledge, building on work from three centuries of mathematicians and hundreds of contributors to Lean and Mathlib. We are optimistic that AI-assisted verification of mathematical proofs will help reduce the burden of refereeing mathematics in an era where more proofs are being produced than ever before.
You can read about the process on our Science Blog: anthropic.com/research/forma…
And see the complete proof on GitHub: github.com/anthropics/fermat…