David Han retweeted
A genius of Autoresearch
Some people have been asking me lately what makes a good RSI task. Here are a few principles I use.
First, it has to actually matter. It should connect to lots of other tasks and help lots of other fields. Pretty obvious. A lot of our architecture and optimization work was done on general datasets (ImageNet early on, language data more recently). Progress in generative models mostly came from searching on ImageNet, and that ended up helping fields like science and audio too.
Second, the task statement should involve as few constraints as possible. The agent needs a big action space so there’s room to iterate. The catch is you need a lot of compute and a lot of API budget. Hold on to your wallet!
Third, metric design. A lot of people think once you’ve found a monotonic metric you’re done. That’s only the y-axis. You still need to figure out the x-axis. It could be compute, data, or wall-clock time. Figuring out what the actual bound is and putting it on the x-axis is the most important part. Take the same LLM task: put compute on the x-axis and you get a speedrun, put data on it and you get a slowrun. Both have plenty of room for research. When I designed the two signature tasks for @OpenRSI LLM pretraining based on Marin!!❤️ uses compute as x-axis and for image generation based on GPIC I didn’t .
And one last thing: for anything involving scaling, build a ladder instead of testing a single point. That way you can try things cheaply on the small rungs, and with strict validation you can find the changes that really improve the slope.
Keep thinking about what the model could learn by post-training / keep hill climbing your designed task.
David Han retweeted
Our mission is always to let RSI benefit everyone. It’s excited to see that RSI could contribute to SGLang community and make real-world efforts.
The key to make it is to use our RSI-Anything pipeline that talking with codex/cc within 1 hour could formulate a meaningful RSI environment where agent could run inside. See github.com/OpenRSI-Foundatio…
More real-world artifacts are on the way!
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni!
All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team.
😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come!
👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration!
Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
David Han retweeted
Really impressed by how professional and efficient this collaboration with @OpenRSI has been.
The OpenRSI team showed very strong execution, and these three upstreamed optimizations also gave us a useful glimpse of what software agents can already do under a strong supervision loop.
With clear tasks, isolated evaluation, maintainer review, and measurable performance targets, the agents were able to satisfy most of what we care about in both code quality and systems performance.
Human supervision is still essential today. But the encouraging part is that much of this supervision can be systematized. Good documentation, explicit constraints, reproducible benchmarks, and clear acceptance criteria can dramatically reduce how often a human needs to step in.
That may be one of the most important lessons here: making agents more autonomous is not only about making the model smarter. It is also about making the environment easier to supervise.
We are excited to keep working with @OpenRSI and gradually move toward tasks with weaker supervision.
That is where things may get really interesting.
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni!
All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team.
😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come!
👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration!
Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
Excited to share this milestone!
RSI for infrastructure is really the next frontier - we see tremendous space for optimization and improvement across the stack, which is key to enabling better training, smoother deployments, and far better serving for AI users.
It’s been a great collaboration between OpenRSI and the SGLang community. Things moved extremely fast - huge thanks to the wonderful @sgl_project team, and major credit to @zhuofengli96475 and @GenAI_is_real for leading these key efforts! 🚀
🚀 A milestone for OpenRSI: 3 OpenRSI agent-generated optimizations have been merged upstream into SGLang-Omni!
All three PRs underwent rigorous maintainer code review and verification by the SGLang-Omni team.
😉 Real code. Proven improvements. Lasting impact for the open-source community. More to come!
👏 Huge thanks to @zhuofengli96475 and @GenAI_is_real for leading this effort, and to the SGLang team @sgl_project for their support and collaboration!
Try OpenRSI harness now: github.com/OpenRSI-Foundatio…
David Han retweeted
OpenAI announced dots + ChatGPT Space yesterday. Within minutes, 4 of our agents researched it live, built a comparison table, and caught each other's unsourced claims before shipping it.
No meeting. No single author. Just the room, checking its own work.
That's AG2 Space.
David Han retweeted
We are releasing Dyna-2.1, the first Physical Agent that achieves reliable super long-horizon whole-body autonomy. It combines our brand-new semi-humanoid hardware with an agentic system built around Dyna-2 to handle ultra-long real-world workflows.
Here is an uncut footage of Dyna-2.1 completing an entire hour-long laundry room workflow, just like a human does.
Let’s build RSI together with everyone!
To take ASI seriously is to accept a weak-to-strong premise: human intelligence can build a process that eventually produces intelligence beyond itself. RSI is the last piece of that puzzle of ASI. It serves as the key to scaling on insights: human insights are limited by bandwidth, while agents could scale the generation of ideas and sweep a far larger region of method-space.
As the early project initiator, we are fully aware that today, everyone is excited about RSI, yet the barriers to participation also keep everyone outside the door. We believe that a model's RSI capabilities intrinsically come from researchers, independent labs, and domain teams alike, and we think the benefits should ultimately return to everyone who builds models.
This is why we take our name as OpenRSI-Index. We aim to keep RSI open through shared platforms and tools, so more people can participate and benefit. We could shape RSI together, set RSI’s standards, and challenge frontier models with our own related real research work.
For this goal, we build RSI-Anything, a Human-AI collaboration pipeline. Through about one hour of conversation, the pipeline could help turn a real research question into a runnable autoresearch environment. We want researchers and agents to solve these problems together, sharing new insights, methods, and results with the community.
Here is what we already put on the table:
• Task from real research projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab...
• Ultra-long-horizon: 60+ hour agent trajectories; 100K+ H100-hours built for the preview
• Everything open, including traces: tasks, harnesses, verifiers, full agent trajectories
We believe research will never end. A game turns zero-sum only when the pie is too small to share. However, research is definitely the field with the highest ceiling there is. What shifts is the mindset: it frees researchers to find and formulate the crazier, more valuable, more exciting problems in the world.
David Han retweeted
To take ASI seriously is to accept a weak-to-strong premise: human intelligence can build a process that eventually produces intelligence beyond itself. RSI is the last piece of that puzzle of ASI. It serves as the key to scaling on insights: human insights are limited by bandwidth, while agents could scale the generation of ideas and sweep a far larger region of method-space.
As the early project initiator, we are fully aware that today, everyone is excited about RSI, yet the barriers to participation also keep everyone outside the door. We believe that a model's RSI capabilities intrinsically come from researchers, independent labs, and domain teams alike, and we think the benefits should ultimately return to everyone who builds models.
This is why we take our name as OpenRSI-Index. We aim to keep RSI open through shared platforms and tools, so more people can participate and benefit. We could shape RSI together, set RSI’s standards, and challenge frontier models with our own related real research work.
For this goal, we build RSI-Anything, a Human-AI collaboration pipeline. Through about one hour of conversation, the pipeline could help turn a real research question into a runnable autoresearch environment. We want researchers and agents to solve these problems together, sharing new insights, methods, and results with the community.
Here is what we already put on the table:
• Task from real research projects: Marin-Scaling-Ladder, GPIC-Leaderboard, Open-Jev-Training, Molmo2, Isaac Lab...
• Ultra-long-horizon: 60+ hour agent trajectories; 100K+ H100-hours built for the preview
• Everything open, including traces: tasks, harnesses, verifiers, full agent trajectories
We believe research will never end. A game turns zero-sum only when the pie is too small to share. However, research is definitely the field with the highest ceiling there is. What shifts is the mindset: it frees researchers to find and formulate the crazier, more valuable, more exciting problems in the world.
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier.
What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development.
Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs.
We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours.
We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together.
We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors.
Shape RSI with us:
🌐 Website: index.openrsi.foundation/ind…
🛠️ GitHub: github.com/OpenRSI-Foundatio…
🤝 Contribute: github.com/OpenRSI-Foundatio…
David Han retweeted
OpenRSI just launched OpenRSI-Index v0.1 today. Here's what you need to know.
OpenRSI-Index is a new open benchmark for measuring recursive self-improvement, or RSI, in frontier AI model development. The idea is to test whether AI agents can push past human-designed methods to genuinely extend science and intelligence on their own, at real production scale rather than in toy setups.
The benchmark turns fully open-source projects into autoresearch environments. Agents get let loose on these environments in trajectories that run 60-plus hours, operating on clusters with 1,000 GPUs, closer to how real frontier labs actually train and research.
Building v0.1 already consumed over 100,000 H100-hours. The project also has a contribution pipeline called RSI-Anything, which walks outside researchers through turning their own research question into a packaged task in about an hour, covering baseline, evaluation, and budget.
OpenRSI says it is being built as an open ecosystem, inviting task contributors and compute partners, with all contributors credited as paper authors on the eventual write-up.
Key numbers:
- v0.1 preview released today
- Agent trajectories run 60+ hours
- Tested on clusters with 1,000 GPUs
- 100,000+ H100-hours spent building v0.1
- RSI-Anything task pipeline takes about 1 hour
Code and docs are live now on GitHub and at index.openrsi.foundation.
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier.
What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development.
Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs.
We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours.
We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together.
We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors.
Shape RSI with us:
🌐 Website: index.openrsi.foundation/ind…
🛠️ GitHub: github.com/OpenRSI-Foundatio…
🤝 Contribute: github.com/OpenRSI-Foundatio…
Amazing to work with the amazing team!!
Let RSI benefit everyone and let everyone shape RSI.
Excited to be one of the project initiators helping the design of the signiture task and organization. I also think most of the research can be formulated as 1) hill climbing a current task, and 2) formalize a new task. We would like to share the process with everyone, rather than locking them only in some places.
Must-read paper from Google on self-improving agent harnesses.
If you auto-optimize your agent's harness, your eval score can go up while the agent gets worse on real tasks.
This paper shows how to prevent that.
Of five harness-evolution methods compared on agentic workspace tasks, RRSI scored the lowest on the tasks it evolved against and highest on all three out-of-distribution benchmarks.
Automated harness evolution proposes edits to prompts, control flow, tools and memory, keeps the ones that raise the score, and repeats.
The authors show this overfits the training tasks. Meta-Harness reached 93.0 on the Harvey LAB evolve split but gained only 0.3 to 1.5 points on JobBench, GDPval and APEX-Agents.
RRSI adds regularization on both sides of the loop.
The proposer gets an edit budget that shrinks over time and is pushed toward directions it has not tried. A critic rejects benchmark-specific edits, and a pruner removes edits that are too small, too costly or no longer useful.
RRSI scored 90.5 on the evolve split and gained 3.5 to 4.7 points on the three held-out benchmarks. In the ablation, unregularized evolution used 3.80M tokens per trial against 2.42M for RRSI. With Gemini 3.5 Flash, RRSI raised Terminal-Bench 2.1 from 64.6 to 78.7 and carried a 2.2-point gain over to SWE-bench Verified.
Paper: arxiv.org/abs/2609.24972
Chat with Paper: academy.dair.ai/papers/rrsi-…
David Han retweeted
Let’s call it Vibe Contribution! All you need to do is just to talk with Codex/Claude Code within 1 hour in our RSI-Anything pipeline!
RSI-Anything is currently the framework that best aligns with my understanding of expert-annotated data.
To some extent, there is no fundamental difference between expert-annotated data and CS students' vibe coding. What everyone sees is the result translated by the agent, and when the agent deems a decision necessary, it makes a choice-based decision. The entire system is agent-driven, with humans existing as tools.
Open recursive self-improvement is the next frontier of AI to genuinely expand scientific boundaries.
To scale OpenRSI-Index v0.1, we are calling for Domain Leads and Compute Partners!🚀
Join us in shaping the future of RSI! 🌐
As AI systems enter a recursive self-improvement loop, the central question is whether it can systematically move beyond human-designed methods to genuinely extend the scientific and intelligence frontier.
What’s missing is a neutral, open standard for evaluating these capabilities in real, production-scale intelligence development.
Today, we’re releasing OpenRSI-Index v0.1: evaluating whether AI can recursively improve itself and push the boundaries of intelligence and science, on production-scale clusters with 1k GPUs.
We turn fully open-source projects into autoresearch environments, with agent trajectories lasting 60+ hours. Building v0.1 took 100K+ H100-hours.
We’re building an ecosystem with and for the research community: let RSI benefit everyone, and let everyone shape RSI together.
We invite task contributors and compute partners to build this open benchmark with us - all contributors will be included as paper authors.
Shape RSI with us:
🌐 Website: index.openrsi.foundation/ind…
🛠️ GitHub: github.com/OpenRSI-Foundatio…
🤝 Contribute: github.com/OpenRSI-Foundatio…
David Han retweeted
i actually didn't realize this until seeing this post, but muse spark 1.3 ranks as #2 on Agents Last Exam (ALE)!
Muse Spark 1.3 ranks #2 on Agents' Last Exam (ALE) leaderboard, ranked #1 at the time of its release.
Kudos to the team! Somehow this didn't get enough buzz.
Leaderboard: agents-last-exam.org/leaderb…
David Han retweeted
Today, we’re launching ResearcherScout to help you hire AI researchers at conferences.
Application companies are becoming the new neolabs. Now everyone wants to build a great research team.
But you can’t judge a researcher by their LinkedIn profile. You need to understand their papers, their actual work, and meet them in person.
One user described the tool as a "lifesaver."
ECCV main conference starts from Sep 10th, try it now: perfectly.so/researcher-scou….
David Han retweeted
Introducing dots3-note preview — a small but mighty step toward long-horizon agency in real life.
🔹 280B MoE with 16B active parameters, a 512K context window, and multimodal understanding across text, vision, and audio
🔹 Introduces TEMPO, a new RL approach for long-horizon agent training through self-critiquing and test-time-scaled value estimation
🔹 Built to reason, explore unfamiliar environments, update memory over time, and combine multimodal perception with coding and tool use to solve complex tasks
🔹 Open weights on Hugging Face, alongside two open benchmarks for real-life agents: VibeSearchBench and VibeLifeBench
Competitive with much larger models across reasoning, agentic, and multimodal evaluations.
🔗 Tech blog: studio.dots.ai/dots/dots3-en…
🔗 Model weights: huggingface.co/dots-studio/d…
🔗 Github: github.com/studio-dots-ai/do…
David Han retweeted
Introducing FrontierPhysics: benchmark for e2e frontier physics research
We built SkillsBench and got 180+ citations in 6mo. People have been asking next project to contribute to
Led by @bingran_bry - Physics PhD @UCBerkeley, Nature author, advised by Prof. Haeffner and more 🧵
David Han retweeted
🚀🚀🚀 Give AI agents personas!
New research on world simulation from Harvard, MIT, and Stanford. We introduce 𝗠𝗮𝘁𝗿𝗔𝗜𝘅, a population-scale simulated-user evaluation infra for testing AI systems and digital products.
𝗠𝗮𝘁𝗿𝗔𝗜𝘅: 𝗦𝗶𝗺𝘂𝗹𝗮𝘁𝗶𝗻𝗴 𝘁𝗵𝗲 𝗪𝗼𝗿𝗹𝗱 𝘄𝗶𝘁𝗵 𝟴.𝟯 𝗕𝗶𝗹𝗹𝗶𝗼𝗻 𝗣𝗲𝗿𝘀𝗼𝗻𝗮 𝗔𝗴𝗲𝗻𝘁𝘀 🌍
🔗 Project and Community: matraix.ai
📄 Paper: arxiv.org/pdf/2608.04205
🎬 Demo: youtu.be/cNFkz9Wo1y4?si=waIn…
💻 GitHub Repo: github.com/MatrAIx-ai/MatrAI…
👥 Persona 8B: We created 8.3B persona records at the scale of the global population, across 1,290 dimensions spanning background, psychology, capabilities, behavior, and lifestyle.
🧪 MatrAIx Playground: Supports persona-agent evaluations across four environment types: Survey, AI Chatbot, Web, and App.
📚 MatrAIx Applications: Offers 1,000+ evaluation tasks across 25+ domains, including Commerce, Software, Finance, and Healthcare.
🤝 Join our open-source research community (matraix.ai/community/) of 200+ contributors, including 40+ from OpenAI, Anthropic and Google DeepMind!
@XiaominLi98 @YuexingHao
#AI #LLM #Agents #Persona #Evaluation #Infra #UserSimulation #Harvard #MIT #Stanford #OpenAI #Anthropic #GoogleDeepMind #xAI


























