Research scientist at Google DeepMind | previously at Microsoft Research and Postdoc at UMass. Views are my own. Most tweets in Mongolian 🇲🇳.

Bay Area
Joined January 2010
Tsendsuren retweeted
Samsung open-sourced a method that shrinks 13B parameter LLM into less than 1 GB. It's called "LittleBit" Instead of storing AI weights as standard numbers, they used latent factorization to crush them down to extreme sub-1-bit levels. In some configurations, they hit 0.1 bits per weight. But here is where the architecture gets crazy. When you compress an AI this much, you can stop doing math. Instead of forcing the hardware to do heavy floating-point multiplication, LittleBit replaces the core computation with a bitwise XOR operation. It swaps complex matrix math for basic sign flips. The results rewrite the rules of model deployment: • Unlocks a massive 11.6x inference speedup relative to standard FP16 models. • Radically reduces memory footprint and loading bandwidth. • Maintains robustness in extreme sub-0.5 bit regimes where previous compression methods catastrophically fail. This is not a clever optimization. It is the blueprint for running massive, state-of-the-art AI locally on cheap, resource-constrained devices.
159
609
132
4,665
230,753
BREAKING: OpenAI’s solution to Navier–Stokes does not match its Lean verification. The most important article to read today is not one of OpenAI’s 700 AI-generated math papers. It is this other paper, making a deep and worrying point: A Lean-verified proof does not automatically validate the proof written in natural language, nor does it mean that the formal statement captures the intended theorem. During translation, an AI can change an assumption, weaken a statement, or replace the argument entirely. It can hallucinate another theorem. Lean correctly verifies the result. But the proved result may no longer be what the paper claims. This is a general problem. Things get spicy when the authors examine OpenAI’s proposed Navier–Stokes solution. They identify at least two mismatches between the written intermediate results and their Lean counterparts: One estimate claims that four additional input derivatives suffice. The Lean version requires five: a weaker result. A pressure-flux estimate is obtained through a different bound, and proved through a different argument. It is not clear whether these mismatches invalidate the entire proof. But they raise an important issue. OpenAI is flooding us with claimed revolutionary breakthroughs. Yet nobody knows whether the proofs are correct or whether they prove what they claim to be proving. Epistemia at scale. * Paper in the first reply
231
921
313
5,370
628,813
This is the way. I am a big believer in hybrid computing. This is much more efficient, accessible and secure.
BREAKING: @JensenHuang just said @nvidia and @Microsoft have "reinvented the personal computer completely." Two things change: 1 - Your computer becomes the home for your agents. 2 - Your AI runs partly on your machine and partly in the cloud. Right now your laptop sits on your desk all day and waits for you. Huang wants it to keep working while you're out. You give an agent a task, and it uses your apps and files on its own. You check in from your phone. He's said he'll text his agents on WhatsApp and they'll text back. The agent lives on a computer you own, at home or on your desk, and runs around the clock. I like the idea. Now I want to see one agent finish one boring, real task for me from start to finish.
144
Tsendsuren retweeted
Today marks a new chapter for Windows, as we bring unmetered intelligence to every desk and every home, and make every PC a place where agents can work securely on your behalf. Some highlights of what we announced: • MAI-Code-1.1 Flash: 137B parameter coding model w/ 256K context window, which is now optimized to run on your PC! • GitHub Copilot now hands off work to local models like MAI-Code-1.1 Flash, helping projects cost a lot less without sacrificing quality. • With Hybrid Intelligence, Copilot can now take action directly on the PC and keep sensitive work on your device. • And with Code in Copilot, you can essentially build any software you need on your desktop, without any cloud token spend, and it’s just super at it. You’re no longer limited to what’s in an app store! Your PC becomes an infinite software factory. • Security is foundational to all this, which is why we are also bringing together Windows and Agent 365 so agents can work within secure boundaries on-device, including MXC a local sandbox for agent execution. Windows becomes your secure agent box! • All this comes to life on a new generation of devices, like Surface Laptop Ultra, powered by NVIDIA RTX Spark. Can’t wait to see what you build with all this.
573
971
313
7,657
759,507
openai/math: 722 papers from an internal OpenAI model, 372 results I built a search engine for it 🧮 • plain English: "how many shuffles randomize a deck?" • names/jargon: "UGC", "Kaplansky" • filter: Lean-verified, negative results, reasoning traces mishig-openai-math-explorer.…
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
3
3
16
1,248
1/ Introducing PhilosophyBench from @StanfordAILab @StanfordHCI, the first independent, large-scale benchmark for evaluating AI’s philosophical capabilities. philosophybench.org
31
101
38
537
58,030
Some RL plots are more fun to watch
Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: mimo.xiaomi.com/rl/
176
IMO Chinese startup ecosystem also deserves some credit here.
Moonshot AI CEO Yang Zhilin surprised some of his Carnegie Mellon professors by choosing to return to China. Now, his Beijing-based startup’s latest private funding round values it at $50 billion. 🔗 on.wsj.com/4A9SV5g
2
422
Ah this is cool
wait DeepSeek is “model merging” during RL?? used across diff Harness/Config checkpoints to combine/init information so we can train on N diff harnesses, take the successful ones, merge those checkpoints, and keep training? wild that this just works well 👀
7
474
Tsendsuren retweeted
Our group discovered that reasoning models produce fractals when asked to solve hard problems. We can use nonlinear dynamics to probe the thinking processes of recurrent depth models on Sudoku, mathematics, and even ARC-AGI (1/N) arxiv.org/abs/2609.04963
113
624
125
4,720
460,818
It took 9 years to go from neural nets designing its architecture to designing an entire experiment. Neural nets designing its architecture to self-improve didn’t work back then.
1
3
284
Tsendsuren retweeted
🚨 Excited to share our new work on WikiSkill (led by the awesome @LiyanTang4)! We propose a method that co-evolves agent skills and a wiki that integrates agent experience into a growing knowledge base. WikiSkill substantially outperforms current SoTA skill evolution methods 👇
Introducing WikiSkill📚: compiling agent experience into persistent knowledge for skill evolution! 🧠 Persistent wiki + evolving skills 📈 Evaluated on 5 benchmarks & 5 models 📊 WikiSkill consistently maintain superior performance
5
10
1
39
6,209
It would be interesting to plug-in linear attention in the middle, something like mid infini-attention (arxiv.org/abs/2404.07143).
Replying to @Muennighoff
we use a simple sliding window that only keeps the most recent tokens + the prefix this makes generation cost constant (capped at prefix + sliding window len) so you can scale indefinitely
1
310
To me, it tells a story about how redundant Transformer layers are - the fact that you can almost swap top layer to bottom and bottom to top and how leaky its representations are perhaps due to the residual connection.
For me, a fun entry-point into this paper is just to observe: The transformer is a silly architecture for token generation. During inference, transformers operate with their hands tied behind their back. They do this because it makes training cheaper.
2
471
Будда-н сургаал ч үгүй, коммунист ёс суртахуун ч үгүй Тэнгэр дээдэсээс ч айдас үгүй … ер н л Ёс үгүй болохоор: үсгүй мич л болдог юм шив .. улам олоноороо
9
145
2
356
8,826
Interesting. Years back in MetaNet exp, I set random weights for each layer but only train their fast-weight counterparts. Since fast-weights are updated based on the random weights grad, I was hoping to see more cross domain generalization. Wasn’t clear on few-shot image tasks
Randomly initialized transformers can do arithmetic and more, sometimes as well as the trained counterparts. (They still train the emb / unemb layers)
2
238
Нутаг ус санагдуулдаг гоё дуу
Энэ хамгийн гоё буриад дуу юм шиг санагдаад байдышт… Дадалд явж байхад нэг ах аймар гоё дуулахыг сонссон… Ерөөсөө санаанаас гардаггүй юм 🚶🏻‍♀️🎧
1
162
Intuition: Compaction is agentic recurrence (RNNs), whereas recursion (RLMs) is agentic attention. Recurrence maintains a constant-size state at all times by compressing the past. Attention keeps all context fully represented and then decides how to re-process it for each step.
21
58
12
599
62,948
Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL Jiabao Ji, Yujian Liu, Li An, Rohit Jain, Gungor Polatkan, Siyu Zhu, Shiyu Chang arxiv.org/abs/2607.25816 [𝚌𝚜.𝙰𝙸]
1
1
444
1/Again about continual learning. What if catastrophic forgetting isn't a bug, but a mathematical necessity? We build complex systems to retain old knowledge, but new theory proves that in non-stationary streams, perfect memory is an optimization liability. Here is why. 🧵
3
32
1
159
11,913