Research Scientist @ ByteDance Seed | Prev. @UWaterloo @MITIBMLab @01AI_Yi @WHU_1893
Joined February 2022
- Tweets164
- Following728
- Followers102
- Likes301
this is open ai
Nearly half a year of silence. We spent it studying one problem: how far RL can scale.
MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks.
Streaming the run: mimo.xiaomi.com/rl/
Xin Yan retweeted
Thinking more about why I feel strongly about this, these AI slowdown discussions make me think of fire. (And I think we should just call it a slowdown instead of "pacing" to stay true to ourselves.)
Fire is dangerous. People can burn themselves and even burn down cities. But imagine if early humans had concluded that fire was too dangerous for ordinary people and should only be controlled by a small group of experts.
I don't want to live in a world optimized for maximum safety. I like adventure, uncertainty, and the freedom to make choices whose consequences I have to bear, including the possibility of hurting myself. That's part of what makes life interesting.
Maybe AI is dangerous in the same way powerful technologies have always been dangerous. But humans are not made of glass. We've lived with fire, electricity, nuclear weapons, and pandemics.
I would rather bear the consequences of my own freedom than be kept safe by someone else deciding what I'm allowed to do.
I'm completely fine with big, powerful players deciding to slow themselves down. That's their freedom. What I hate is when they ask governments to step in and slow everyone else down too, where they start interfering with others' freedom. It almost reminds me of being a kid playing soccer. Another kid complained to the teacher, and then the teacher banned everyone from playing. I hated that feeling then, and I guess I still do: if you don't want to play, fine, but why do you want nobody else to be able to?
I feel like this is always one-way preaching. Anthropic comes across as incredibly arrogant to me. You keep telling us what responsible AI development should be, and now even laying out a plan for how the entire field should proceed. But do you actually listen to what other people think, especially those who disagree with you?
As a paying Anthropic customer, when I need help, I get a bot. Honestly, even my phone company feels easier to get through to. You claim to care deeply about humanity, but you don't seem to even care about your own customers.
And if you truly believe slowing down serves humanity, you don't need anyone's permission. Just slow down yourself. Anthropic is one of the companies pushing the frontier forward. Raise less money, train fewer frontier models, release them less aggressively. If Anthropic slows down, AI development slows down.
My hope is in open source and hardware. I think one day everyone should be able to have a powerful AI in their pocket, and bear the legal and societal consequences of what they choose to do with it. I'd much rather have that future than one where a few companies become our AI overlords, babysitting the rest of us because that's supposedly "safer" for us and apparently they understand what's best for humanity better than the rest of us do.
it’s awesome. so sad that I am not part of the game now. got me rethinking what really matters.
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Xin Yan retweeted
Wow, this brought back a project Woojeong Kim, @srush_nlp, and I worked on in 2024.
A strong model read a task description, turned it into "ability vectors", and passed them to a weak model.
Sasha even came up with a great name for it:
Strong-to-Weak Ability Transfer (SWAT) 😆
We never published it, but the idea eventually developed into ProgramAsWeights, which compiles task descriptions into small reusable neural programs that run locally.
Very cool to see @aimalysheva and the Mostik team make a related idea work so well at this scale.
programasweights.com
meet @mostik_ai!
what happens when you put 12 PhDs in one room for four months? first place on the ARC-AGI leaderboard, which I can't say much about while the competition is still running. and this, which I can.
everyone's arguing about whether open models will catch up to frontier models. we think it's the wrong question. here's the one we pose: why does a frontier model have to generate your answer at all, when the only thing you need from it is the reasoning?
we do this by enabling models to communicate in latent space. through our protocol, hidden states pass straight from a frontier model into a small one running on your infrastructure -- no text between them, and neither model is fine-tuned. two models from different families, sharing reasoning, both left untouched.
how do we know it works? we tested it on a setup where a 753B model reads the problem, and a 4B edge-class model writes the answer. with this approach, we get results 80% as accurate as the frontier model, but at 20x faster performance.
we're committed to preventing frontier model lock-in and are already partnering with inference providers to accelerate open-weight adoption. we've done this between 15 of us, in four months, 12 PhDs and a Fields medalist, backed by @generalcatalyst
WIRED has the first external account of the company and the work: wired.com/story/russian-star…
full writeup, the setup, and all the numbers: mostik.ai/read-more
This is exactly what I needed! I’ve already installed it in Cursor.
RAP checks whether the agent actually follows my Cursor Rules. Like I want agents to use my custom Parquet writer instead of writing their own, since they always put the entire dataset into a single row group.
Frustrated your coding agent keeps ignoring your rules?
What if every rule could take on a life of its own?
I built RulesAsPrograms which turns each rule into a function that checks agent's work.
Instead of hoping agent follows rules, rules check agent
github.com/programasweights/…
Xin Yan retweeted
1/
AI coding agent 每次 grep 都在把整个仓库重读一遍。
snapgrep 给 Pi 加了个 trigram 索引：17MB 仓库，147.7ms → 2.1ms。
结果和 ripgrep 逐字节一致，无后台进程，索引比源码还小。
github.com/Owen718/snapgrep
I often say this to myself, too. Now I'll try to solve the Riemann hypothesis tonight.
8 days ago, while jogging, I asked Claude to solve the Riemann Hypothesis
It didn’t. 1.5 days later, it proved >= 67% of the zeros are on the line (prev: 41.6%)
Still not sure what that means, but some analytic number theorists seem excited
anthropic.com/research/riema…
Xin Yan retweeted
We live in a multimodal world. We see, talk, act, and dream.
Yet most LLMs still start with language pretraining. Why not train them natively with multimodal I/O from scratch?
Because it’s SUPER HARD, adding modalities triggers training instability, design complexity, and often modal competition
So what’s the path forward?
Introducing: Towards Physics of Multimodal Pretraining (junlinhan.github.io/projects…)
We unpack the underlying mechanics of multimodal pretraining across 4 aspects: Knowledge Flow, Modality Synergy, Early Unification, and Recipe.
shit what's going on there
Announcing Discovery Loop!
I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
♾
Learn more at: discoveryloop.com
Xin Yan retweeted
I know this is supposed to be the dystopian future, but our ProgramAsWeights (programasweights.com) is basically this idea: compiling function descriptions directly into runnable neural programs (LoRA adapters), skipping code altogether.
The part I find most exciting isn't replacing code, but implementing fuzzy functions that are easy to describe but hard to write using code, like repairing broken JSON, classifying if a message is urgent.
Xin Yan retweeted
this moment would not be complete without uncle jurgen coming in like a ray of sunshine rising from the sands of arabia
forever a jurgen stan 🫡
fun fact: distillation was published in 1991, then copied by others xitter.cf/SchmidhuberAI/status/1…
lmao
hello there the jacobian conjecture is false thanx to my close friend akhil for asking about it and my other close friend fable for working during the world cup final
((1+xy)^3 z + y^2 (1+xy) (4+3xy), y + 3 x (1+xy)^2 z + 3 x y^2 (4+3xy), 2 x - 3 x^2 y - x^3 z): \C^3\to \C^3, has jacobian determinant -2, and sends (0, 0, -1/4), (1, -3/2, 13/2), and (-1, 3/2, 13/2) to (-1/4, 0, 0)
Xin Yan retweeted
怀念在 GPT-3.5 时代写代码的日子。
那时，LLM 还只是一个朴素的对话框，但我却每天都能学到新东西，感觉自己在实实在在地变强。不再需要自己翻文档、读报错、反复调试。它把那些繁琐、枯燥、耗神的部分统统挡在外面，只把思考和设计的乐趣留给我。看着它在浏览器里一行行吐出的代码和解释，我常常带着一种近乎孩子气的满足。那时的 AI，既提高了效率，也守护了创造的快乐。而如今的 coding agent，把效率提升了十倍甚至百倍。它几乎接管了整个流程，却把我从创造的核心里彻底摘除。我的思考速度已跟不上它输出、执行、测试、修复的节奏。望着那些命名规范到近乎冷酷、结构整齐得像被算法亲自梳理过的配置文件和代码，我知道自己完成了更多的工作，却再也找不到曾经那种「我正在亲手把想法变成现实」的喜悦。我得到了大量产出，却仿佛失去了创造本身。
Xin Yan retweeted
I’m an AI PhD at Oxford wanted to move to the US for years, but it never worked out. Maybe I can share why.
To me, it feels like the US is making it harder and harder for Chinese talent to come or stay.
First, Trump signed PP10043, which meant I couldn’t go to the US for any graduate school. That’s why I had to turn down my Stanford offer and come to Oxford.
Later, I qualified for an EB-1A (the extraordinary ability green card). But because I was born in China, I’m stuck in a country-specific backlog that could take another 4–5 years.
Meanwhile, on the other side, I’m getting 3–4 emails from China almost every week. They offer high salaries, free housing, generous research funding, and even offered an astonishing amount of money just for me to come back and have a conversation.
I’ve always loved the American spirit. I truly believe my abilities could have a bigger impact there. But honestly, the whole process has been incredibly frustrating.
All of this makes me wonder whether I should just give up.
Why am I spending so much energy trying to go to a country that doesn’t seem to want me, when another country is doing everything it can to bring me back?
it's a good model
Today, we are introducing Inkling.
Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available.
thinkingmachines.ai/news/int…
Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
