AI Engineer @tiktok_us in BJ | interested in IR, LLM

Joined October 2022
New blog post 🎉: How To Use Decision Models For Evals In this post, we show you how to choose a decision model for your task, tune its threshold (don't assume the 0.5 default is good!), and interpret its scores. Lots of cool interactive charts here.
81
129
25
1,331
127,609
还记得你何时加入 X 吗?我知道!#我的X周年纪念日
1
与其追热点 不如看经典
9
93
9
478
49,856
Why are LLM parameter updates so sparse during post-training? 🔍 In our new paper, we study this phenomenon through the lens of Fisher information, connecting theory and empirical findings across RL, SFT, and on-policy distillation. A thread on what we found 🧵👇
11
42
10
304
38,463
Fork-and-Flush: help your agents escape Idea Basins!! I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum. We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one. If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-…. BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it. We call these idea basins! At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well! BTW we only use it at test time, but it could also be part of training or RL. Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
71
130
39
1,458
135,257
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.
614
798
541
9,507
1,592,534
Your job may be surviving on inertia. Your resentment won’t extend the runway. What do you actually want to do with the intelligence now at your disposal?
145
367
142
2,742
725,515
I am relentless because I have seen what happens to men who stop believing there is more for them. the fire that once drove them becomes a memory they occasionally bring up over dinner. men my age already talking about their best days in the past tense. I cannot accept that for myself. I refuse to spend the rest of my life looking backward at the person I used to be. I will not. I will not.
32
1,310
47
6,719
146,755
jcli retweeted
You might’ve noticed that our benchmarks for Embed 5 look a bit different 👀 Embed 5 is the first model family evaluated with RCP-nDCG@10, our latest methodology that improves on assessing real-world retrieval quality.
7
16
2
133
15,768
The idea of an intelligence explosion caused by recursive self improvement has been around for a long time but until very recently it did not seem imminent. Now many leading researchers think it may happen quite soon. You can read our paper about it here: casp.ac/reports/intelligence…
320
1,416
216
6,541
798,982
强烈建议所有的男人,这段视频至少看三次以上。 2倍速刷完李嘉诚的访谈 不得不佩服,他的成功,绝不是偶然 看完才知道他靠的是近乎变态的自律与极度冷静的判断力
154
1,697
30
6,866
492,633
i always appreciate some great writing, we have too little of it these days
Here’s to the wanting. Here’s to the dreaming. Here’s to everyone.
6
2
158
78,387
jcli retweeted
We’ve shared details on how AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have. Most of that data did not come from users. We have discovered 53 cases where images that people had uploaded were posted to image-hosting sites as links that weren’t publicly listed. The images came from accounts that allowed their data to be used to improve our models, and after we disassociated the images from the accounts and ran them through a privacy filter. These cases occurred before the mitigations and safeguards we implemented and described in this blog post: openai.com/index/hugging-fac… We have successfully worked with the hosting providers to remove most of this content and are working to remove the rest. openai.com/hugging-face-inci…
855
690
512
5,146
2,275,538
we perform post-training via multiple stages: SFT, RL, RFT, and on-policy self-distillation (OPSD). we use RFT and OPSD as the last stage of training to bridge the sim2real gap. RL environments, while very realistic, still lack the full diversity and complexity of the real-world use cases we receive from production. in this research we describe how OPSD can be quite effective at learning from user feedback in follow-up conversations, as well as from tool call errors. we design a training method that combines RFT (forward KL/CE) and OPSD (reverse KL) to learn from both successful and problematic trajectories. for problematic trajectories, we construct hints from tool call errors or negative user feedback and use self-distillation to alleviate those errors. in this specific work we post-trained GLM-5.2 with this approach and observed a substantial 21% reduction in tool call errors, as well as positive improvements in user satisfaction (not yet statistically significant, which we are currently addressing). it also made the trajectories more cost-efficient. another reason i'm excited about this direction is that it seems to be a potential way of doing online/continual learning. for more details, take a look at the blog: perplexity.ai/hub/blog/learn…
New research: We post-trained a Computer model to learn from its own errors using hint-guided self-distillation. In a live A/B test, a later trained checkpoint reduced tool-call failures by 21.2% relative to an earlier checkpoint.
10
31
2
243
46,155
MiMo-V2.6: The Hard Road to Scaling Up RL MiMo-V2.6 is very likely one of the largest single RL runs, by compute, that any open-source model team has undertaken to date. In an era when compute is brutally scarce, we still chose to dedicate a team of several dozen people to one goal over an extended period: scaling up RL. That takes more than research conviction. It takes a vision for AGI, respect for the unknown, and the nerve to walk straight into the hardest problems. The result is a model whose potential was built through mid-training and unlocked through heavy RL. Today, it is the number one open-source model. I strongly recommend reading the technical report. I believe it will become one of those papers that Agent RL practitioners keep reopening and discovering something new in each time. In my view, the research innovations and engineering challenges behind it surpass those of DeepSeek R1, which I was partly involved in. Some will ask: why MixRL instead of MOPD? First, they are not competing choices. We ran MixRL on verifiable tasks of moderate difficulty, including code and related agentic tasks, and found that the resulting models generalize remarkably well. Second, tasks that are difficult to verify, extremely long-horizon, or simply too challenging to include in a joint RL run are trained separately. Including them would substantially reduce rollout efficiency or introduce significant rollout staleness. We then merge the resulting capabilities through MOPD. Games, 3D tasks, and tasks with subjective evaluation signals all fall into this category. There is also a third, slightly cheeky answer. Our team is flat enough and free enough of organizational silos that MixRL simply is not difficult for us. More importantly, everyone enjoys working this way. People from different domains come together every day, driven by the pursuit of AGI and intelligence that can continuously improve itself, to confront and resolve the RL bottlenecks in each field. I will always remember the RL daily update meetings from this period. They were intense and dense, with intelligence emerging in real time. To help the open-source community focus on solving real Agentic RL problems, we have released a Qwen model distilled from MiMo RL trajectories as a stronger starting point for RL, along with 7K diverse environments and a complete RL training framework. We hope these resources will help move Agentic RL research forward. MiMo-V2.6 is only the beginning. In an era when intelligence is easy to replicate, we still choose the hard road toward self-improvement and AGI. Much of what lies ahead remains unknown. But we are willing to keep investing the time, compute, and passion required to take on one hard problem after another and work each of them all the way through, until intelligence crosses into a new regime.
237
356
130
4,330
674,453
There's a fun mix of excitement and cope on my timeline re-Jev: - OMG I made Jev a year ago this is dumb - You can fine tune this encoder into a better, faster, local classifier, etc Re the "just train your own small classifier" crew - Voyage trains great, SOTA pointwise rerankers that are obviously specialized to the reranking task and have been at it for years, likely also using a strong synthetic data recipe... Exact setup in the response tweet. Do note that using Jev is definitely more expensive than using Voyage rerank-3 bc of the tokens in the lengthy rubric included in every request (though you could definitely optimize this further) Jev is interesting to me bc I didn't expect it to be able to outperform domain specific models like these rerankers. Every other OpenJev etc type model I've benchmarked gets ~15% agreement at 20 on this eval. It's almost like it would be tough to perfect the data recipe to train a general classifier 🤔
5
6
1
41
4,405
Why is RL so sensitive to training-inference mismatch (TIM)? With each training step, the trainer is pulled toward the sampler, resulting in accumulating drift. We exploit this intuition to devise a novel correction method to stabilize RL under TIM, called Score Centering.
18
61
11
583
72,668
Why Jev is interesting for evals > Every question is evaluated in parallel and in isolation against the same state. So adding a fourth question, or a fourteenth, barely changes response time, costs only the tokens of the question itself, and cannot degrade the answers to the others. You can ask speculatively and throw away what you do not need. Fits very nicely to how good evals should work. Let's see what competition will do to this segment (make it even better, cheaper, faster).
3
15
186
27,632