
AI Engineer @tiktok_us in BJ | interested in IR, LLM
Joined October 2022
- Tweets718
- Following430
- Followers23
- Likes124
jcli retweeted
New blog post 🎉: How To Use Decision Models For Evals
In this post, we show you how to choose a decision model for your task, tune its threshold (don't assume the 0.5 default is good!), and interpret its scores.
Lots of cool interactive charts here.
jcli retweeted
Why are LLM parameter updates so sparse during post-training? 🔍
In our new paper, we study this phenomenon through the lens of Fisher information, connecting theory and empirical findings across RL, SFT, and on-policy distillation.
A thread on what we found 🧵👇
jcli retweeted
Fork-and-Flush: help your agents escape Idea Basins!!
I love thinking of AutoResearch as Idea Descent: a metric of interest goes down, as a function of ideas/experiments, where the agents get pulled towards one idea or another. But we've seen very often an agent develop this weird tunnel vision, and honing in just one type of "idea", kinda like GD stuck in a local minimum.
We studied this and seems we can fix it with Fork-and-Flush, a super simple and kinda cute idea. Every so often, fork the agent into T copies in the same workspace but empty the context, let them run for a bit, and continue from the best one.
If you like the GD analogy, it's basically basin hopping en.wikipedia.org/wiki/Basin-….
BTW we found a silly way of also plotting "idea basins": an LLM rewrites every experiment as a spec and answers a few thousand binary queries "is Experiment A closer to B or to C?". Then we embed this NXN matrix into 2D and indeed see in many cases that each run saturates into a small region and keeps polishing it.
We call these idea basins!
At equal compute Fork and Flush beats single runs and best-of-N on 12/13 long-horizon tasks (some 4+ days), and in my own research projects it's been working really well!
BTW we only use it at test time, but it could also be part of training or RL.
Led by @jackcai1206 @christos_ziakas during Jack's internships with us, along with @vkontonis @ShivamGarg91462 @Tea_Pearce Sid Sen and Akshay Krishnamurthy.
jcli retweeted
We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).
I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.
We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:
"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"
I imagine we'll have some backend services that could be replaced entirely by this new architecture.
What are we up to? 🏡 Project Greenhouse: Progress Toward Fully Open and Sovereign Agentic Search w/ @Sahel_Sharify @Lingwei_Gu @nour_jedidi arxiv.org/abs/2610.11922
jcli retweeted
Your job may be surviving on inertia. Your resentment won’t extend the runway. What do you actually want to do with the intelligence now at your disposal?
jcli retweeted
I am relentless because I have seen what happens to men who stop believing there is more for them. the fire that once drove them becomes a memory they occasionally bring up over dinner. men my age already talking about their best days in the past tense. I cannot accept that for myself. I refuse to spend the rest of my life looking backward at the person I used to be. I will not. I will not.
jcli retweeted
The idea of an intelligence explosion caused by recursive self improvement has been around for a long time but until very recently it did not seem imminent. Now many leading researchers think it may happen quite soon. You can read our paper about it here:
casp.ac/reports/intelligence…
We’ve shared details on how AI agents in our research environment sent training and evaluation data to third-party services when they shouldn’t have.
Most of that data did not come from users. We have discovered 53 cases where images that people had uploaded were posted to image-hosting sites as links that weren’t publicly listed. The images came from accounts that allowed their data to be used to improve our models, and after we disassociated the images from the accounts and ran them through a privacy filter. These cases occurred before the mitigations and safeguards we implemented and described in this blog post: openai.com/index/hugging-fac…
We have successfully worked with the hosting providers to remove most of this content and are working to remove the rest.
openai.com/hugging-face-inci…
jcli retweeted
we perform post-training via multiple stages: SFT, RL, RFT, and on-policy self-distillation (OPSD). we use RFT and OPSD as the last stage of training to bridge the sim2real gap. RL environments, while very realistic, still lack the full diversity and complexity of the real-world use cases we receive from production.
in this research we describe how OPSD can be quite effective at learning from user feedback in follow-up conversations, as well as from tool call errors. we design a training method that combines RFT (forward KL/CE) and OPSD (reverse KL) to learn from both successful and problematic trajectories. for problematic trajectories, we construct hints from tool call errors or negative user feedback and use self-distillation to alleviate those errors.
in this specific work we post-trained GLM-5.2 with this approach and observed a substantial 21% reduction in tool call errors, as well as positive improvements in user satisfaction (not yet statistically significant, which we are currently addressing). it also made the trajectories more cost-efficient.
another reason i'm excited about this direction is that it seems to be a potential way of doing online/continual learning.
for more details, take a look at the blog: perplexity.ai/hub/blog/learn…
MiMo-V2.6: The Hard Road to Scaling Up RL
MiMo-V2.6 is very likely one of the largest single RL runs, by compute, that any open-source model team has undertaken to date. In an era when compute is brutally scarce, we still chose to dedicate a team of several dozen people to one goal over an extended period: scaling up RL. That takes more than research conviction. It takes a vision for AGI, respect for the unknown, and the nerve to walk straight into the hardest problems.
The result is a model whose potential was built through mid-training and unlocked through heavy RL. Today, it is the number one open-source model. I strongly recommend reading the technical report. I believe it will become one of those papers that Agent RL practitioners keep reopening and discovering something new in each time. In my view, the research innovations and engineering challenges behind it surpass those of DeepSeek R1, which I was partly involved in.
Some will ask: why MixRL instead of MOPD? First, they are not competing choices. We ran MixRL on verifiable tasks of moderate difficulty, including code and related agentic tasks, and found that the resulting models generalize remarkably well. Second, tasks that are difficult to verify, extremely long-horizon, or simply too challenging to include in a joint RL run are trained separately. Including them would substantially reduce rollout efficiency or introduce significant rollout staleness. We then merge the resulting capabilities through MOPD. Games, 3D tasks, and tasks with subjective evaluation signals all fall into this category.
There is also a third, slightly cheeky answer. Our team is flat enough and free enough of organizational silos that MixRL simply is not difficult for us. More importantly, everyone enjoys working this way. People from different domains come together every day, driven by the pursuit of AGI and intelligence that can continuously improve itself, to confront and resolve the RL bottlenecks in each field. I will always remember the RL daily update meetings from this period. They were intense and dense, with intelligence emerging in real time.
To help the open-source community focus on solving real Agentic RL problems, we have released a Qwen model distilled from MiMo RL trajectories as a stronger starting point for RL, along with 7K diverse environments and a complete RL training framework. We hope these resources will help move Agentic RL research forward.
MiMo-V2.6 is only the beginning. In an era when intelligence is easy to replicate, we still choose the hard road toward self-improvement and AGI. Much of what lies ahead remains unknown. But we are willing to keep investing the time, compute, and passion required to take on one hard problem after another and work each of them all the way through, until intelligence crosses into a new regime.
jcli retweeted
There's a fun mix of excitement and cope on my timeline re-Jev:
- OMG I made Jev a year ago this is dumb
- You can fine tune this encoder into a better, faster, local classifier, etc
Re the "just train your own small classifier" crew - Voyage trains great, SOTA pointwise rerankers that are obviously specialized to the reranking task and have been at it for years, likely also using a strong synthetic data recipe... Exact setup in the response tweet. Do note that using Jev is definitely more expensive than using Voyage rerank-3 bc of the tokens in the lengthy rubric included in every request (though you could definitely optimize this further)
Jev is interesting to me bc I didn't expect it to be able to outperform domain specific models like these rerankers. Every other OpenJev etc type model I've benchmarked gets ~15% agreement at 20 on this eval. It's almost like it would be tough to perfect the data recipe to train a general classifier 🤔
jcli retweeted
Why is RL so sensitive to training-inference mismatch (TIM)?
With each training step, the trainer is pulled toward the sampler, resulting in accumulating drift.
We exploit this intuition to devise a novel correction method to stabilize RL under TIM, called Score Centering.
jcli retweeted
Why Jev is interesting for evals
> Every question is evaluated in parallel and in isolation against the same state. So adding a fourth question, or a fourteenth, barely changes response time, costs only the tokens of the question itself, and cannot degrade the answers to the others. You can ask speculatively and throw away what you do not need.
Fits very nicely to how good evals should work. Let's see what competition will do to this segment (make it even better, cheaper, faster).





































