@FCHEN_AI

LLM reasoning MTS@Microsoft Superintelligence Team PhD @Stanford

Joined July 2022
  • Tweets13
  • Following202
  • Followers86
3Media
Excited and grateful to have contributed to reasoning, RL, and STEM climbing behind the MAI-Thinking-1 model. This has been an intense but fun hill-climbing journey from scratch. Please check out the technical report: microsoft.ai/wp-content/uplo…
3
115
The best part of this job is seeing students graduate and launch their careers! Congrats to Feng Chen, Atsushi Yamamura, Tamra Nebabu, Linnie Wharton and Daniel Kunin. They are all going on to top positions across artificial intelligence, medicine, and physics. Proud of you!
1
1
2
90
6,170
Proud to be part of the team behind this new open-source SOTA formal math prover! 🚀 Achieving 72.95% on MiniF2F with simple BFS strategy. Our models are trained using expert iteration and DPO, pushing the boundaries of formal theorem proving. 📄 Paper: arxiv.org/abs/2502.03438

BFS-Prover: Scalable Best-First Tree Search for LLM-based...

Recent advancements in large language models (LLMs) have spurred growing interest in automatic theorem proving using Lean4, where effective tree search methods are crucial for navigating the...

arxiv.org
Ran Xin
@RanXinByteDance25 Feb 2025
🚀 Excited to announce BFS-Prover, our state-of-the-art theorem proving system in Lean4! We've achieved 72.95% on the MiniF2F test, surpassing all previous systems including DeepSeek-Prover-v1.5, InternLM2.5-StepProver, and HunyuanProver 📈 🔥 Key innovations: - Simple Best-First Search, rather than complex MCTS - No critic model (value function) needed Our 7B tactic generation model is now available on huggingface: huggingface.co/bytedance-res… Paper: arxiv.org/abs/2502.03438 This challenges the perceived necessity that complex search methods are necessary for formal theorem proving. Sometimes simpler is better! @FCHEN_AI
2
247
1/ Our new paper: “Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning” on how to change training to better exploit test-time compute! co-led by @AllanRaventos, w/ Nan Cheng, @SuryaGanguli & @ShaulDr arxiv.org/abs/2502.07154
1
5
1
18
9,125
more replies
3/ We propose directly optimizing for coverage in the fine-tuning loss with Direct Coverage Optimization (DCO). DCO attenuates gradients on high-confidence samples, regularizing away from overconfidence. We demonstrate superior accuracy frontiers over CE loss in MATH and MiniF2F.
1
1
1
237
4/ We extend our algorithm to automated theorem proving and math QA with CoT. In theorem proving, our approach improves performance by controlling the exploitation and exploration tradeoff in proof trees. In CoT, where overconfidence is less severe, we also see performance gains.
1
2
173
Join us at the ML for Multiscale Processes workshop at #ICLR2025 to hear from our three first amazing keynotes: Qianxiao Li blog.nus.edu.sg/qianxiaoli/ Sergei Gukov theory.caltech.edu/~gukov/ Charlotte Bunne aimm.epfl.ch/
2
1
147
Come check out our #NeurIPS2024 spotlight poster on feature learning tomorrow! 📍East Exhibit Hall A-C #2102 📅Thu 12 Dec 4:30 p.m. — 7:30 p.m. PST
Daniel Kunin@KuninDaniel26 Sep 2024
🌟Announcing NeurIPS spotlight paper on the transition from lazy to rich🔦 We reveal through exact gradient flow dynamics how unbalanced initializations promote rapid feature learning co-led @AllanRaventos and @ClementineDomi6 @FCHEN_AI @klindt_david @SaxeLab @SuryaGanguli
7
48
10,064
Want to learn about SGD's implicit bias towards simpler subnetworks generated by permutation symmetry?! Come to our NeurIPS poster session tomorrow morning 10:45 - 12:45 Hall B1+B2 (level 1) #906
Daniel Kunin@KuninDaniel22 Jun 2023
Our new preprint reveals how SGD biases neural nets towards vastly simpler subnets w/ superior generalization via stochastic collapse to invariant sets & explains why prolonged large learning rates help co-led w/ @FCHEN_AI @atsushi_y1230 & @SuryaGanguli arxiv.org/abs/2306.04251
1
7
75
14,048
1/ Our new paper lead by @AllanRaventos @mansiege , @FCHEN_AI asks when in-context learning of regression can solve fundamentally *new* problems *not* seen during pre-training, and reveals it as an emergent capability arising from a phase transition... arxiv.org/abs/2306.15063

Pretraining task diversity and the emergence of non-Bayesian...

Pretrained transformers exhibit the remarkable ability of in-context learning (ICL): they can learn tasks from just a few examples provided in the prompt without updating any weights. This raises...

arxiv.org
4
39
2
167
31,960
Excited to share our new work on how pretraining task diversity affects in-context learning.
Mansheej Paul
@mansiege17 Jul 2023
Can in-context learning learn new tasks different from those in the pretraining data? Is this an emergent ability, i.e. does it arise from pretraining without being explicitly optimized for? How does this depend on pretraining task diversity? 🧵 1/ arxiv.org/abs/2306.15063

Pretraining task diversity and the emergence of non-Bayesian...

Pretrained transformers exhibit the remarkable ability of in-context learning (ICL): they can learn tasks from just a few examples provided in the prompt without updating any weights. This raises...

arxiv.org
4
330
Excited to share our new paper on how SGD biases towards simpler models via stochastic collapse to the invariant sets.
Daniel Kunin@KuninDaniel22 Jun 2023
Our new preprint reveals how SGD biases neural nets towards vastly simpler subnets w/ superior generalization via stochastic collapse to invariant sets & explains why prolonged large learning rates help co-led w/ @FCHEN_AI @atsushi_y1230 & @SuryaGanguli arxiv.org/abs/2306.04251
2
4
850
Load more