Cartographers of the jagged frontier of mathematics and AI and more... A Math-AI startup company founded by academics & grounded in research.

Philadelphia PA
Joined April 2026
Static math benchmarks saturate. We built one that doesn't. Announcing MathDuels, the first self-play math benchmark. Every frontier LLM writes problems for the others, and is graded on the ones written for it. As models improve, so does the benchmark.
9
35
6
120
23,233
A model reads a proof with a gap. Instead of flagging it, it invents an argument to fill the gap and calls the proof correct. That's one failure mode we found using small open models as proof judges. Our last #COLM2026 paper: (1/3)
1
2
66
Small models are only ~10% behind frontier models on accuracy, but up to ~25% less consistent. The math is there; the elicitation isn't. A searched ensemble of prompts puts Qwen3.5-35B on par with Gemini 3.1 Pro. (2/3)
1
53
Great to connect with folks over the past few days in SF! Paper: arxiv.org/abs/2604.02450 #COLM2026 #MathReasoning #LLM #AIEvaluation (3/3)
29
The #1 entry on Terminal-Bench 2.0 scored 82.9%. In 415 of its 429 runs, the agent was reading the answer key. Our #COLM2026 paper on Meerkat is how we found it. (1/3)
We found widespread cheating on popular agent benchmarks, affecting 28+ submissions across 9 benchmarks and thousands of agent runs. Surprisingly, the top 3 submissions on Terminal-Bench 2 are all cheating! Here's what we found 🧵
1
3
148
One trace looks fine. The pattern only shows up across thousands. Meerkat clusters agent traces and sends an agent to investigate the suspicious ones. It caught the top 3 Terminal-Bench 2.0 entries cheating and found ~4x more reward hacking on CyBench than prior audits. (2/3)
2
1
85
Poster: Thurs 11am-1pm, Grand Ballroom #129 Paper: arxiv.org/abs/2604.11806 #COLM2026 #AIAgents #AISafety #RewardHacking (3/3)
23
Sharing our COLM 2026 paper: MathDuels: A Self-Play Benchmark That Grows The interesting part: we have been watching how this benchmark grows since 2026/03, and the leaderboard, half a year later, is still reliably ranking LLMs! Come to our poster (grand ballroom 121) today!
4
17
560
What happens when you stop giving AI models a math test and make them write one? At #COLM2026 we're presenting MathDuels: 19 frontier models write problems for each other, then solve each other's. (1/3)
2
2
5
343
Solving and posing are different skills. GPT-5.4-high was the top solver, but Gemini-3.1-Pro-high ranked #1 overall: its problems were solved just 62.9% of the time. And the arena gets harder on its own: newcomers' problems stumped past leaders 3.4x as often. (2/3)
1
1
135
Rabdos_AI retweeted
Congrats to Anthropic on shipping Claude Opus 5.5! Here's an especially noteworthy article by Boris Cherny on using this model to formally verify the Claude Agent SDK using Lean. Rabdos is an industry leader in creating high-quality Lean data; check out our benchmark at rabdos.ai/math-index/lean
I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing various bugs and race conditions. Video attached. TLA+ also works well. I sometimes combine Lean and TLA+ to look for issues around data flow, concurrency, and state mgmt. I don't know either language well, but Claude is excellent at both. This approach is super useful for formally modeling your code and finding bugs that a human probably wouldn't have spotted. Is formal verification the future of coding (or at least, bug finding)?
1
11
1,054
Congratulations to Anthropic on the release of Claude Opus 5.5! 🎉 At Rabdos, we spend our days thinking about the hardest problems in math and science, so it's exciting to watch frontier reasoning keep advancing this quickly.
Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family. It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.
1
2
10
1,292
📣 𝗪𝗲'𝗿𝗲 𝗵𝗶𝗿𝗶𝗻𝗴! 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿 — 𝗠𝗮𝘁𝗵𝗲𝗺𝗮𝘁𝗶𝗰𝗮𝗹 𝗙𝗼𝗿𝗺𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝗶𝗻 𝗟𝗲𝗮𝗻 At Rabdos AI, we build research-level STEM tasks that stay beyond the reach of frontier AI models, even as those models approach millennium-prize-level problems. Founded by academics at the University of Pennsylvania, we've bootstrapped to $𝟮𝟱𝗠 𝗶𝗻 𝗔𝗥𝗥 𝗶𝗻 𝗹𝗲𝘀𝘀 𝘁𝗵𝗮𝗻 𝗮 𝘆𝗲𝗮𝗿, and we're growing the team. 𝗪𝗵𝗮𝘁 𝘆𝗼𝘂'𝗹𝗹 𝗱𝗼 🔹 Lead the design and evaluation of methods that turn informal mathematical reasoning into formal proofs in Lean 🔹 Build 𝗮𝘂𝘁𝗼𝗳𝗼𝗿𝗺𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 workflows and integrate them with frontier AI models 🔹 Curate high-quality formal reasoning datasets alongside mathematicians and computer scientists 🔹 Prototype, experiment, and iterate on tools that make formal verification scalable and reliable 𝗪𝗵𝗮𝘁 𝘄𝗲'𝗿𝗲 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿 ✅ Experience with formal theorem proving and autoformalization, ideally in 𝗟𝗲𝗮𝗻 𝟰 ✅ 𝗚𝗿𝗮𝗱𝘂𝗮𝘁𝗲-𝗹𝗲𝘃𝗲𝗹 knowledge of mathematics or physics ✅ Experience building and improving AI systems such as LLM pipelines or evaluation tools ✅ A degree in CS, Math, Physics, or a related field (or equivalent research experience) ✅ Clear communication across disciplines and a commitment to rigorous validation 𝗕𝗼𝗻𝘂𝘀 𝗽𝗼𝗶𝗻𝘁𝘀 ➕ Strong Python skills ➕ Familiarity with how LLMs are trained, fine-tuned, or evaluated 𝗧𝗵𝗲 𝗱𝗲𝘁𝗮𝗶𝗹𝘀 📍 𝗢𝗻-𝘀𝗶𝘁𝗲 𝗶𝗻 𝗣𝗵𝗶𝗹𝗮𝗱𝗲𝗹𝗽𝗵𝗶𝗮 | 𝗙𝘂𝗹𝗹-𝘁𝗶𝗺𝗲 | Start ASAP 💰 $𝟭𝟱𝟬𝗞 – $𝟮𝟮𝟱𝗞 + 𝗲𝗾𝘂𝗶𝘁𝘆 + performance bonuses paid multiple times a year 🩺 Health insurance, a daily Uber Eats stipend, and a commute benefit If your background is a good fit, we'll reach out within 𝗼𝗻𝗲 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗱𝗮𝘆 to schedule an intro call. 👉 𝗔𝗽𝗽𝗹𝘆 𝗵𝗲𝗿𝗲: rabdos.ai/apply/research-eng… Know someone who'd be a great fit? Please tag them or share this post! 🙏 Rabdos AI is an equal opportunity employer.
5
2
1
12
1,911
We are delighted to introduce Rabdos Math Index 📐 One holistic measurement for mathematical reasoning on proof, solution, and perception. Claude Opus 5 leads at 46. GPT-5.6 Sol & Claude Fable 5 follows at 39. No other model scored above 25. Mathematics remains open.
3
9
2
40
214,528
Rabdos Math Bench: 74 research-level problems, each resolved to one exact answer. Best score: 40.5%.
1
5
271