Pinned Tweet
Superintelligence isn't about discovering new things; it's about discovering new ways to discover
I think our latest work formalizes Meta Chain-of-Thought which we believe lies on the path to ASI
When we train models on the problem-solving process itself—rather than the final solution—they internalize how to think about reasoning tasks, not just what to think
The next wave of AI is a Meta-CoT loop. We can't predict what novel forms of thinking might emerge, but it points to an extraordinary synthetic future
I'm so proud of @synth_labs team & our incredible open science collaborators for getting this work out
we’re at #COLM2025🍁 come see our poster # 26 (session 1) today
reach out ✉️ if you'd like to chat!
nathan lile retweeted
🚨🚨New Paper: Training LLMs to Discover Abstractions for Solving Reasoning Problems
Introducing RLAD, a two-player RL framework for LLMs to discover 'reasoning abstractions'—natural language hints that encode procedural knowledge for structured exploration in reasoning.🧵⬇️
nathan lile retweeted
much more convinced after getting my own results:
LoRA with rank=1 learns (and generalizes) as well as full-tuning while saving 43% vRAM usage! allows me to RL bigger models with limited resources😆
script: github.com/sail-sg/oat/blob/…
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
thinkingmachines.ai/blog/lor…
nathan lile retweeted
I’ll take the opposite view - current methods are saturating and we need at least 1 practical breakthrough and at least two fundamental ones (which will likely take years) just off the top of my head to reach AGI. None of these are oversight or safety related.
nathan lile retweeted
NEW: Is the internet changing our personalities for the worse?
Conscientiousness and extroversion are down, neuroticism up, with young adults leading the charge.
This is a really consequential shift, and there’s a lot going on here, so let’s get into the weeds 🧵
nathan lile retweeted
Really great collaborating with @NathanThinks!
Reach out if you're working on synthetic data generation, offline RL, or simulating agentic behavior.
Earlier this year we partnered with SynthLabs (synthlabs.ai/), a post-training research lab, to generate a 351 billion token synthetic dataset 10x faster and 80% cheaper.
Read more in our case study: sutro.sh/case-studies/synthl…
nathan lile retweeted
you have no idea how hard it is to get an rlhf model to be even “centrist” much less right reactionary. they must have beat this guy up pretty hard
nathan lile retweeted
I’m not big on identities, but I am extremely proud to be American. This is true every day, but especially today—I firmly believe this is the greatest country ever on Earth. The American miracle stands alone in world history.
I believe in techno-capitalism. We should encourage people to make tons of money and then also find ways to widely distribute wealth and share the compounding magic of capitalism. One doesn’t work without the other; you cannot raise the floor and not also raise the ceiling for very long.
The world should get richer every year through science and technology, but everyone has to be in the “up elevator”. I think the government usually does a worse job than markets, and so we need to encourage our culture of innovation and entrepreneurship. I also believe that education is critically important to keeping the American edge.
I believed this when I was 20, when I was 30, and now I am 40 and still believe it. The Democratic party seemed reasonably aligned with it when I was 20, losing the plot when I was 30, and completely to have moved somewhere else at this point. So now I am politically homeless. But that’s fine; I care much, much more about being American than any political party.
I’d rather hear from candidates about how they are going to make everyone have the stuff billionaires have instead of how they are going to eliminate billionaires.
The American experiment has always been messy. I am hopeful for another great 250 years. Happy 4th!
nathan lile retweeted
Xiaomi got 200,000 orders in 3 minutes for the YU7 and I’m not even surprised.
The value proposition is just nuts.
I’m kinda of bummed because it means a few more years of having to satisfy demand from China before global expansions.
nathan lile retweeted
the future is about smart tokens
What if models could learn which problems _deserve_ deep thinking?
No labels. Just let the model discover difficulty through its own performance during training.
Instead of burning compute 🔥💸 on trivial problems, it allocates 5x more on problems that actually need it ↓
Our new method (ALP) monitors solve rates across RL rollouts and applies inverse difficulty penalties during RL training.
Result? Models learn an implicit difficulty estimator—allocating 5x more tokens to hard vs easy problems, cutting overall usage by 50%
🧵👇1/10

ALT Pareto efficiency analysis reveals how models distribute computational resources across problems of varying difficulty. Cumulative token allocation curves for problems ordered from easiest to hardest, aggregated across MATH-500, OlympiadBench, and AIME datasets. Shaded regions indicate easy (0-50%) and hard (80-100%) problem ranges. Just Enough Thinking: Efficient Reasoning with Adaptive Length Penalties Reinforcement Learning https://arxiv.org/pdf/2506.05256
nathan lile retweeted
China is winning the race to Type 1 Civilization and we're not even aware it's happening.
By 2030, China will have the manufacturing capacity to build an entire U.S. worth of generation from solar and storage alone - every single year.
The flow of energy is what drives physical change in the world. Control over that flow is power. Sunlight on Earth is 10,000x as powerful as all human energy sources combined.
This is the Space Race of the 21st century, and it has the potential to lift humanity into an unprecedented era of abundance, but it will dramatically shift the balance of power globally.
The West needs to wake up and realize that the construction of TW scale solar manufacturing and TWh scale battery manufacturing is not about climate change.
Sunlight is >99.9% of Earth's energy budget. The tools that harness the power of the Sun are the tools that unlock power at planetary scale.
This is not a race that we can afford to ignore.
Check out our latest pod with @JessePeltan, which is just 3 hrs straight of him dropping bangers like the one below
nathan lile retweeted
Check out our latest research on data. We're releasing 24T tokens of richly labelled web data. We found it very useful for our internal data curation efforts.
Excited to see what you build using Essential-Web v1.0!
nathan lile retweeted
congrats @rm_rafailov on your hard-earned acceptance to the USofA as alien of officially extraordinary ability. The alien piece comes as no surprise to your mates of course, but at least the general public now has fair warning and a fighting chance. To celebrate with a fitting observation from a preeminent Kazakh journalist:
"My country send me to United States to make AI model. Please, come and see my model. If it not success, I will be execute."
🇺🇸🇺🇸🇺🇸🇺🇸🇧🇬🇧🇬🇧🇬🇧🇬
nathan lile retweeted
When we first published our work on this 9 months ago it was rejected for being impractical in realistic cases. Six months later it was rejected for lack of novelty. It’s the way academic publishing goes.
Generative Reward Models impact compounds daily.
way stronger interest now than when we published last fall 👇
many excellent recent extensions—cool seeing where
researchers take GenRM
we bootstrapped our way to generalized meta-reasoning capabilities with generative reward models
classical reward models can be worse than random on new reasoning tasks 🎲
we see improvements in robustness, generalization, interpretability and an opportunity to unify RLHF/RLAIF
...it's rationales all the way down 🐢
nathan lile retweeted
I was going to call this dumb, but former NTSB board member John Goglia just texted me and told me to reply with this instead:
The issue raised in The Rehearsal is whether the authority gradient affects copilots' willingness to assert themselves at critical junctures and captains' willingness to hear copilots in those moments.
Neither the question nor the answer dealt with this well-recognized cockpit issue. Nor the NTSB's analysis of the issue of what was lacking in current training and its recommendation of role-playing to ameliorate this situation.












































