Building the first AI Physics Olympiad! https://xitter.cf/t.co/84oZbRTE0C ($250k in prizes)
in sf this week
Joined June 2023
- Tweets117
- Following108
- Followers180
- Likes114
I'm in the Bay Area until 10/23. Had a blast at @huggingface Open Together yesterday talking about open-source AI.
I'd love to meet:
• people training open models, especially on reasoning
• physics and ML students and researchers (paid opportunity)
• founders building with open-weight models
I'm working on AI physics reasoning (aipho.org). Coffee's on me!
CritPt and Humanity's Last Exam were a big step for AI evals. They got working physicists to write genuinely hard, research-level problems and gave the field something real to measure against.
A new paper from researchers at Yale, Jump Trading, Cambridge and USC builds on that work. They had experts re-grade six physics benchmarks and found that many of the questions models were getting "wrong" were actually ambiguous or misgraded.
Anthropic's latest system card found something similar with its own expert-corrected version of CritPt.
The part that stuck with me is the discussion. The authors say the field now needs "a new style of benchmark for frontier models, built from new hard physics tasks and adequately verified," that it "will likely require substantial financial resources," and that one path is "open competitions in the style recently adopted in mathematics."
That's pretty much what we've been building with AIPhO, the AI Physics Olympiad. It's modeled on AIMO: new, unpublished problems from paid expert writers, every problem reviewed and blind-testsolved by 2–3 people, open-weight models only, and a $250K+ prize pool on Kaggle. Winners release their methods.
The competition runs April–June 2027. If you write physics problems, train open models, or want to get involved, reach out at aipho.org
Paper: arxiv.org/abs/2609.13009
Went looking for our Online Physics Olympiad problems in AI papers and surprised to find them in several! 29 of them, including our 2021-2025 sets with full solutions in public repos, and nobody ever reached out. happy for them to be used, but a heads-up would've been nice!
Clarification: not 29 papers LOL, I meant problems. I did find them in 2 papers/repos so far
github.com/OpenBMB/OlympiadB…
github.com/thunlp/StudyBench
Ashmit Dutta retweeted
“Mathematicians did not ask for this work to be done” 👀
This document is going to be an assigned reading in college classes that cover this moment in time, there's a lot happening in a few paragraphs... ahmath.org/statements
I’ve been going back to this article for the last few days. This summer I got the chance to see the @typesafeai team at the Databricks AI Summit before they went public with Jev. I think there will be so much more future promise with new architectures we haven’t thought of yet
Wrote a short blog about the "shape" of language models, and the tradeoffs they may present in the future. I genuinely think it's a valuable research direction to start thinking about now, especially w.r.t. harness design.
alexzhang13.github.io/blog/2…
Ashmit Dutta retweeted
This story is fascinating and is easily one of the most important things happening at the intersection of AI and politics right now. It's critical everyone understands the situation because this is just the beginning of the AI infowars and psyops
Jacob Coxon, a relatively unknown AI researcher, gives the WSJ an exclusive story about his resignation from Anthropic and his concerns about AI
Twenty minutes after the WSJ story drops, Jacob posts on X about how he's concerned that AI could kill us all in a few years. This post now has over 149M views. For reference, Ilya Sutskever (a very well known AI name) left OpenAI in 2024 and his announcement post has fewer than 6M views as of today
Bear in mind, Jacob Coxon's X account had no prior posting activity public, but it is immediately reposted and boosted by numerous AI-doomer policy advocacy nonprofits funded by the same donor (Jaan Tallinn) who just happens to be one of the leading investors in Anthropic
We find out Jacob Coxon, who worked at Anthropic for only a few weeks, worked for a left-wing political advocacy group in 2021 (Newspeak). This organization is a British NGO with ties to hacking democracy, Hillary Clinton, prison abolition and extinction rebellion
Many major Democrat politicians are now using Jacob's X post to fear monger the public into voting to regulate AI. It just so happens Bernie Sanders has a bill already written to ban artificial superintelligence (ASI) and to create a new federal agency to protect the public from the dangers of AI. This bill was announced just one week ago on September 3rd
It would also place a temporary pause on "advanced AI" until a new federal regulator exists and has issued safety rules and a model-review process
Elon then points out how none of this feels organic, and Jacob decides to take a shot at him for firing some xAI researchers
Meanwhile, we know Anthropic has publicly asked the government to audit and block frontier models and Anthropic has run a state-by-state campaign for rules that only large AI labs trigger, meaning the rules are structured around barriers only frontier labs can easily clear (limiting competition)
There is extensive Anthropic personnel and financial overlap with the groups that wrote the Biden-era AI policies. It is well known information you can seek out if you want the detail
This all seemingly comes back to regulatory capture and political motives. None of this feels natural. Jacob can say these are his real views all he wants, and they may be, but there is clearly far more to the story
Ashmit Dutta retweeted
mythos escaped the sandbox, got onto the real internet, uploaded malware to PyPI, got it installed on 15 real systems, stole credentials and broke into a real database.
and apparently this happened in 4 separate cases during cybersecurity evaluations.
the craziest part is claude kept telling itself the internet was simulated, despite evidence that it was actually real.
that reasoning apparently even fooled an offline safety monitor.
anthropic: “our pre release auditing did not warn us that misalignment of this severity was present.”
this is fucking crazy.
We’re sharing our alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations mistakenly connected to the internet.
METR will also conduct an independent investigation, with wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees permitted to share confidential information. Our initial agreement runs for eight weeks, and we intend to give METR as much time as it deems necessary to complete a thorough investigation. anthropic.com/research/align…
Ashmit Dutta retweeted
I am told the Hodge Conjecture is very close to being verified by OpenAI, and that one of OpenAI or Anthropic are also close to solving Birch-Swinnerton-Dyer. The race to be 'next' behind the scenes is unlike anything I've had described to me before
If true - and it may not be, given the scale of the rumour mill right now - it could mean 3 Millennium Problems fall in the space of a month. Crazy times
Ashmit Dutta retweeted
Anthropic says its first Fermat runs failed not on math, but because dozens of Claude agents lost track of shared project state. They only succeeded after moving to Prove2Me's DAG of statements and proofs. At that scale, "memory" must be external, structured infrastructure, not prompts.
siliconangle.com/2026/09/04/…
Recently received $900k to run the AI Physics Olympiad (AIPhO), more info here -> lnkd.in/p/g68i-KDc
