patomp retweeted
INTELLECT Policy Optimization (IPO)
In async RL, trainer and inference never agree exactly on a token's probability. IcePop masks on the ratio, which misjudges both tails
IPO masks on the probability difference instead
patomp retweeted
interesting tech report, especially the distillation part! it's the first time i see a full KL objective (you train on minimizing the distribution of the student with the "approximate" teacher) and then full CE for the last 5%
it's also co-distillation, which means that you actually use the logits from the forward pass that the teacher (here muse spark) produces during training and not the final model after training. does this also mean that you need the same data schedule? there is quite a big student-teacher gap so would be interesting to see if this kind of "smoother" learning reduces that
Today we published a technical report on Muse Glimmer, our 30B open-weight model built for agentic tasks on consumer hardware.
It runs at practical speeds on a Mac or PC with a single consumer GPU at 4-bit precision, and leads comparably-sized open models on tool use and multi-step planning.
Read about the model architecture, distillation recipe, quantization and more: go.meta.me/4Bcvxol
patomp retweeted
Cool results making auto-research / AI-based optimization better by having agents forget! Turns out that just running an agent for a long time can lock it into small steps in the same direction.
patomp retweeted
The AI Scientist v3 is out: The Agentic Scientist.
GitHub: github.com/SakanaAI/AI-Scien…
Blog: pub.sakana.ai/agentic-scient…
arXiv: arxiv.org/abs/2610.18933
The Agentic Scientist is the 3rd iteration of our automated AI Scientist research system, and the shift this time is from “one agent writes a paper” to “a scaffold of parallel agents orchestrates the full research loop.”
The system now orchestrates a pool of parallel workers that share memory and distilled conclusions. Crucially, it features Evidence Generation: isolated agents automatically run controlled baselines and ablations to prove why an idea actually works.
It works as a co-scientist. We used a development version as a co-scientist to explore biologically plausible learning rules. The agent handled the empirical heavy lifting, proposing mechanisms and running experiments. We stayed in the loop to steer direction, verify results, and moderate claims. That feels like the right division of labor for where the technology actually is.
This resulted in a new biologically plausible learning algorithm, and the resulting paper was accepted at ALIFE 2026 as an oral presentation.
We are also releasing AutoIllustrate. It generates editable draw.io vector figures instead of uneditable, hallucinated raster images. Real scientists need vectors, not pixels. I am an SVG junkie ;)
Incredible work by Robert Lange, Luca Grillotti, Yutaro Yamada, Rujikorn Charakorn, and the team!
The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery
Blog: pub.sakana.ai/agentic-scient…
GitHub: github.com/SakanaAI/AI-Scien…
arXiv: arxiv.org/abs/2610.18933
Since 2024, we’ve used The AI Scientist as a way to test how far AI alone can take scientific research, from v1 to v2 to the paper published in Nature earlier this year. The Agentic Scientist is the next step in that line of work, built for the era of coding agents.
Coding agents such as Codex and Claude Code now carry out long software tasks on their own. Handing them a research problem, however, tends to produce a strong hyperparameter optimizer rather than a scientist: scores go up, but ideas stay narrow, and the reasons for the gain are rarely examined.
The Agentic Scientist makes the research loop explicit. It wraps off-the-shelf coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how results are handed to the next stage.
Parallel workers test hypotheses against rubrics defined before any experiment runs, share successes and failures through a common memory, and then ablate the winning program to generate evidence.
We used an early version of the system to study Error Diffusion, a biologically motivated alternative to backpropagation. Our researchers set the question and the rubric, chose directions at each stage, and verified the results, references, and claims before submission. The resulting paper was accepted as an oral presentation at ALIFE 2026. In autonomous runs, the system also raised Doc-to-LoRA retrieval accuracy from 52% to 98% on multi-needle tasks and designed a learned evolution strategy that improves over its CMA-ES reference on unseen tasks.
Alongside it we release AutoIllustrate, a pipeline for scientific figures that keeps an editable draw.io file as the source of truth, so every diagram can be corrected by hand.
Humans define the question and the standards of evidence; agents expand the experiments researchers can run. We call this Co-Discovery, and we are releasing both systems as open source for the community to build on.
patomp retweeted
People often ask me how to properly learn machine learning.
Rule 1: Stop consuming low-quality content such as most YouTube videos.
Rule 2: Start reading rigorous textbooks used in serious MSc programs in machine learning.
Christopher Bishop’s Pattern Recognition and Machine Learning is an excellent choice, and it is freely available here:
microsoft.com/en-us/research…
patomp retweeted
Today we release 𝚍𝟷-𝚞𝚕𝚝𝚛𝚊, our most powerful decision model.
> built for hard decision problems.
> d1-ultra is multimodal
> supports 44 languages
> #1 on the @ArtificialAnlys Decision Index (65)
> leads all eight categories of our 112-evaluation suite
1/
patomp retweeted
OPD forgets less than SFT, yet breaks under teacher mismatch and on long rollouts. Why? Our answer: they optimize different targets. With multiple teachers, SFT (forward KL) learns their arithmetic mixture; OPD (reverse KL) learns their normalized geometric mean. 🧵
patomp retweeted
We now have state-of-the-art pre-trained base models in the 3T-parameter weight class, built entirely in-house at MAI.
Post-training results coming soon -- stay tuned!
Lots of details in our technical blog post.
microsoft.ai/news/frontier-p…
patomp retweeted
We are hiring for our growing RL Environments team at Cohere. We build replicas of real-world enterprise setups across many industries, put our AI agents in them and watch the agents train and improve.
If you know anyone excited to work at the intersection of RL x Data x Engineering, send them our way! Or if that's you, join us: jobs.ashbyhq.com/cohere/dc52…
patomp retweeted
the most interesting paper I read this month.
sparsity comes from on-policy.
(which matches my intuition but this research explains why)
patomp retweeted
BTW tinker is hiring researchers & infra experts!
research: jobs.ashbyhq.com/ThinkingMac…
systems: jobs.ashbyhq.com/ThinkingMac…
Dimuon is a new optimizer we built for LoRA fine-tuning. In our SFT tests, it often reaches lower loss in fewer steps than AdamW. We're still experimenting with Dimuon; you can Tinker with it today and share your feedback!
See our optimizers guide at tinker-docs.thinkingmachines…
patomp retweeted
Deep learning is often taught as a sanitized collection of facts and math. But work at the frontier of AI is messy and empirical.
Our new class, Deep Learning Alchemy, aims to teach the art and science of deep learning in the hands-on spirit of Language Modeling From Scratch.
Students build intuition by running experiments on small LMs, then test it through prediction quizzes: given an unseen code diff, can you predict what happens?
deep-learning-alchemy.github…
My co-instructor @kothasuhas and I think there's significant demand for classes that treat students as serious learners who want to be assessed for what they’ve learned, and so we're trying a new exam-heavy approach to the class (7 exams, 85% of the grade, with the remaining 15% in-person discussion assessments).
We’ve designed the class so students can use AI agents to run their experiments and do their assignment, but they then have to demonstrate that mastery on exams without access to the assignment, and any partial credit on the exam has to come through citations to the assignment, incentivizing them to know what’s in their write-up.
The same assignments and quizzes are public for anyone to try (lecture recordings will be released through @StanfordOnline with some delay). We have some very fun assignments and quizzes covering both classic topics (like NQM, run-to-run stochasticity, and scaling laws) and new and timely topics like muP or hyperball. Made by some real experts like @TristanThrush and @wen_kaiyue
Try some quizzes below!
patomp retweeted
We're hiring across many roles at Trillium Labs, @trillium_labs, to build the open scientific ecosystem for LLMs. Most important hires right now (ideally with start dates soon):
* Full time post-training researchers
* Engineers with experience with popular open-source libraries (megatron, MILES, sglang, Tinker, etc.)
* Mission-aligned generalists, non-profit ops, etc.
* Student researchers
I'm also personally interviewing for MATS right now, which will be very aligned research.
Please reply, email me, DM me, etc AND submit the form on our website.
Excited to build this team. All early hires will work directly with me on all sorts of fun puzzles to rapidly scale our impact.
patomp retweeted
Is on-policy sampling always good for distilling any teacher–student pair?
We study 17 model pairs, and find that a simple alternative—distilling on offline rollouts from the initial student (which we call Semi-OPD)—can often work better and faster than OPD in 14/17 cases.
The gains are strongly correlated with initial teacher–student alignment. When poorly aligned, online student rollouts can degrade and drift away from regions where the teacher provides useful supervision, while initial student rollouts can provide a cleaner and stable signal.
Excited to share my internship paper at NVIDIA! More results and analysis in the paper: arxiv.org/abs/2610.11291
patomp retweeted
"DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning"
Looped Transformers can struggle to combine facts because intermediate hidden states aren't properly aligned with token embeddings.
DiscoLoop fixes this by mixing decoded token embeddings back into the hidden states between loops.
This boosts out-of-distribution accuracy on synthetic two-hop reasoning from below 10% to nearly 100%, while also improving real-world language modeling benchmarks.
alphaxiv.org/abs/2607.00341
patomp retweeted
We're hiring exceptional engineers to join the team.
No degree required. All we care about is you being smart and able to solve hard problems.
Total comp is $550k-$1.1M (job ad says something else, but ignore that).
Join Polaris, a new @GoogleDeepMind team designing the complex evaluation frameworks that will define the future of AI coding agents, and give you the autonomy to prototype architectures and publish research.
Experience? We care about how you think and build, not your “level.”
Apply now: goo.gle/4AD2av6
patomp retweeted
1/
During OLMo3-32B pre-training, accuracy on an arithmetic diagnostic hits 81% at 2.17T tokens, collapses to 0% at 2.19T, and jumps back to 81.7% at 2.21T.
All while validation loss decreases smoothly.
The phenomenon: mode-hopping. 🧵
patomp retweeted
Between this and Apple's recent work, I'm excited to see renewed interest in weight-merging.
I've been interested in trying it out before but have never been able to find a comprehensive study.
How should models continually learn from new data?
SFT on heavily post-trained models often forgets and fails to generalize, so many works try to make the data more on-policy, e.g., on-policy self-distillation, Pedagogical RL...
If you've tried, you might find it’s tricky to balance on-policyness and data quality. What if on-policy isn't necessary? We show that off-policy training actually works well for continually learning, if done right!
🧵1/7
patomp retweeted
Frontier labs are rumored to be adopting looped Transformers for their potential to unlock implicit reasoning. But can looped Transformers really master it?
We find a bottleneck that keeps looped Transformers from perfect implicit reasoning. Using mechanistic interpretability, we show how looping helps implicit reasoning, why it still struggles to generalize, and how to fix it.
We introduce DiscoLoop: a residual design for looped Transformers that improves implicit reasoning.
🧵 Long thread ahead 👇