Infra @FireworksAI_HQ, https://xitter.cf/t.co/LkqyNDPcga CS @iitbombay, @uchicago. Poker. RL and Continual Learning research.
San Franscisco
Joined November 2020
- Tweets397
- Following2.1K
- Followers726
- Likes7.1K
With the increasing adoption of task-specific model routers, we will see many more verticalised, Jev-like classification and regression models, each trained to outperform their general-purpose counterparts on specific tasks.
Auto-research systems are going to come with minimum guarantees / SLAs in the future once enough data is collected on the system's performance
The future of auto-research systems demand models to be time-aware or budget aware.
Enterprises would be allocating a budget to the auto-research system (could be $$, GPU hours or time) and the autoresearch agent would have to utilise this budget judiciously. Currently models are not even close to budget-aware decision making since they can't predict how much of their budget is going to be consumed by a particular experiment.
Great blog and work!
💥New blog post: Are LLM agents time-aware? Can they predict wall-clock time of tasks? Can they estimate time they spent?
We study this on a range of tasks, including long-horizon ones (ProgramBench, PaperBench, DeepSWE, etc).
Joint work with my MATS mentee @MOfengenden!
auto-research is going to require highly scalable code storage infrastructure.
Running experiments at scale and continuously learning from them will generate an enormous amount of code and artifacts that need to be stored and managed. they would also need to provide versioning tools and would have to be much faster than GitHub.
All companies building this will boom
one of the biggest problems in automating training is teaching models to have "taste" in data mixes.
Frontier models right now lack the knowledge of what good data looks like. We would need stronger scaling laws for this so models don't have to spent too many $$ in just running data ablations.
Beautiful paper on RSI!
RSI will not just be internally for the frontier labs, but will come to the enterprise very soon where frontier models will train specialized task-specific models.
A lot of infrastructure will be built just to enable this for the enterprise. Particularly -
1. Auto-research harnesses
2. Aggregation of research so auto-research agents can learn from past experiments
3. Cheaper training
With the smartest person I know, @HarshSensei, we're building evsys-sdk for the community, this open-source repository allows anyone to build their own continual learning system with first-class support for @FireworksAI_HQ.
We believe continual learning will not be a single algorithm, but rather an auto-research system that looks at production traces, figures out the best way to update the model weights by conducting experiments on both data and algorithms, and runs evaluations to measure the results of those experiments.
You can now create a system with one deploy command which pulls in your production traces with some frequency, triggers the ML auto-research loop automatically, which uses any Tinker-compatible training API (@FireworksAI_HQ) to conduct experiments and train your model from the deployed checkpoint, and finally updates it with the better model on inference providers. RL, SFT, self-distillation, custom algorithms, etc. are tools that the auto-research system has at its disposal.
One of the major issues blocking auto-research for ML experimentation is that training code requires too much plumbing, which fills up the agent's context and is non-standardized across experiments, making it difficult to compare experiments and learn from past experiments.
The SDK provides an auto-research harness which takes away the complexity of training code and squashes each experiment down to just a config file, while giving full freedom to run any data or algorithm experiment. This way, the auto-research agent doesn't suffer from context bloat and performs much better than standard auto-research harnesses.
The SDK is also highly extensible and customizable, providing simple abstractions for curating any form of data, running any algorithm, pulling from any trace database, and running training on any Tinker-compatible backend.
We see auto-research as the only solution to continual learning, and this open-source contribution as a way to democratize access to it and push the usage of open-source models.
As a roadmap to enable ML auto-research and continual learning, there is a lot of work to be done:
1. Indexing previous experiments and existing plethora of research to give to the auto-research agent so it keeps improving and learns from past experiments.
2. Making app-layer integrations, e.g. one-click distillation from your traces generated through Claude Code, Cursor, etc. into an open-weights model.
3. Improving the auto-research harness and adding more integrations.
Great talk! The penetration of post-training is slowly increasing, and in the next 5 years, a lot of traditional enterprises will also start owning their models.
As the infrastructure around post-training matures and providers start giving SLAs around guaranteed improvements on their use cases, adoption will increase!
The main guarantee required is that retraining and continual learning on your data should happen without human intervention and be much cheaper than full training, while maintaining the same accuracy. Only then will new model releases from the labs not beat your owned, post-trained model.
Want world class research capabilities, but don’t have the resources of a big lab?
At our recent Sovereign AI event, @gabepereyra shared @harvey ’s “moneyball” approach.
Here’s the playbook:
00:00 Introduction
00:37 Building a research lab on a budget
02:28 Legal Agent Bench, contracting, and the diligence dataset
03:57 Domain experts guiding synthetic data generation
05:23 Why Harvey open sourced its datasets
06:55 Working with the neo labs – and why more than one
08:20 Post-training in-house: building "Associate 1"
09:44 The model serving matrix: 60 countries, fallbacks, SLAs
11:05 Deciding what stays in production
12:29 Simple open source switches and model routing
13:55 Moneyball: "If we win on this budget, we change the game"
14:53 Q&A: Training with sensitive data
17:16 Q&A: Competing for research talent
18:46 Q&A: Designing rubrics that actually challenge frontier models
20:19 Q&A: Where the pipeline breaks — data, research, or infra
22:59 Q&A: The tension in open sourcing a benchmark
25:02 Q&A: Biggest remaining open problems
27:10 Q&A: Competing with horizontal products
A little late, but this is my favourite research on data recipes that go into building frontier models. Auto-research systems in the future will conduct detailed and thorough data ablations in order to curate the right data for training. More $$ would be spent on running data research than algorithm research for enterprise post-training, and this is the perfect use case for the most powerful and frontier models. This would soon convert to the curation of data from production traces.
The paper says the data source and the mix matter more than anything else, and something counter-intuitive was that stronger models don't always lead to better teachers.
Really great work by all the authors! @NeginRaoof_ @RichardZ412 and team.
One of the major hurdles in solving continual learning through auto-research is the creation of high quality training data and evaluation benchmarks from production traces itself. These systems would have to spend a lot of compute and $$ in running ablations across various data curation recipes.
Different algorithms require different characteristics in the training data and that would have to be curated from the ever-changing production traces. The curation recipes itself would have to be adapted on the fly by these auto-research systems. Not all data is trainable.
PostTrainBench traces show us the models are not very good at conducting data ablation experiments.
I analysed AI-native companies from the largest open-source US unicorn companies database by @IlyaStrebulaev (been one of my favourite accounts all these years)
1. Not surprisingly, 22 foundational model labs own 62% of all AI-native unicorn value. ($1.94T of $3.15T). Anthropic and OpenAI are $1.82T of that between them
2. Time to $1B varies by category, model labs and chip companies lead with median ~ 1 year, chips and AI cloud 3, robotics and defense 4, healthcare and legal 5-6, customer service and workflow 7, sales and marketing 8.
3. Defense went 1 unicorn in 2023, 1 in 2024, then 12 in 2025. Robotics went 0 in 2023, then 16 across 2024-26. Anduril alone ($61B) is 47% of the entire defense category's value. Shows a heavy focus on defense and robotics.
Infrastructure companies lead application layer companies (with the exception of coding agents!).
A very interesting paper. On-policy distillation has been getting all the hype and one of the major failure reasons of this algorithm is when the teacher policy is too "far" from the student policy. When applied to multi-turn use-cases the differences add up and the student-policy starts disagreeing with the teacher policy by a large margin. But often the disagreement is large only because the teacher has never been in the student’s state, so the update is noise, not signal.
I've been experimenting with various ways of mitigating this and this paper does something really smart! To figure out if the teacher is genuinely teaching the correct action, this paper adds the teacher action instead of the student action to the student trace and sees whether in the future the trace leads to a success and only then updates the student with this teacher supervision.
Would love to apply this to more use-cases, paper of the month?
Agentic on-policy distillation has a temporal problem: a teacher can look better at the current turn and still push the student into a worse future.
FutureBridge-OPD intervenes at high-disagreement states, restores the same state, rolls the student forward again, and keeps guidance only when the paired continuation becomes more teacher-preferred.
Look Ahead Before You Distill: Future Trajectory Validation of Teacher Guidance for Agentic On-Policy Distillation
Paper: arxiv.org/abs/2608.01953
For post-training/evals, and for bringing auto-research to the enterprise, we need to build exact replicas of the environments agents will act in during production, for every use case and a large part of this environment will be internal tools. Is this automatable by coding agents? How are people thinking about it?
With the cost of software generation going down, there is a whole new class of companies possible that wasn't possible before. These are the vertically integrated companies that own and optimise the end-to-end chain and have lesser dependencies on other businesses/softwares. These companies can build much better products for certain use-cases. Tesla has been doing it for years (owning software, hardware and compiler), a lot more such companies will spring up.
For example, inference companies can optimise all the way down to PTX (compiler), in the future maybe even software-hardware co-design.
Robotic companies will design the sim stack, the perception model, the planner and the actuators against each other.
The best companies are going to start owning things that are critical to their core-business helping them increase their value prop, earlier considered upstream.