THU 24, now UIUC CS PhD

Joined May 2023
🔥 Excited to share Agent World Model in the Wild (WildAWM)! 🌍 Fully open-source synthesis 🔓: No proprietary models needed! We synthesized 341 high-quality environment–task pairs, addressing 3 challenges in task synthesis: ✅ Solvability: At least one agent can successfully complete each task. 🔥 Difficulty: Challenging even for frontier models, with GPT-6-Astra achieving only ~72.7 pass@8. 🎯 Verifier reliability: 95% agreement between task verifiers and agent-as-a-judge ensemble. 📈 Effective RL training: Trained exclusively in these synthetic worlds, Qwen3.6-35B-A3B achieves an average +10.58-point improvement across 8 OOD benchmarks!
8
47
6
159
12,695
Jiarui Yao retweeted
🚀 Introducing Lean4AgenticRL framework. The first method to apply formal verification in an open-ended domain for agentic RL. We apply the Lean4Agent(xitter.cf/RickyRDWang/status/206…) framework to provide process rewards for agentic RL. We train LAR-9B/27B using Lean4AgenticRL. The LAR-9B models achieve a 20.2% improvement on SWE-Bench-Verified, bringing the score to 64.2%. The LAR-27B model achieves 77.3% on TerminalBench-2.1, improving the base model by 7.7%. The LAR-9B/27B outperforms all the known baselines of similar size in SWE-Bench-Verified, TB-2.0, and TB-2.1. Welcome to read our paper and try our model! [1/n] Paper: github.com/RickySkywalker/Le… Model: huggingface.co/amazon/LAR-27… and huggingface.co/amazon/LAR-9B Code: github.com/RickySkywalker/Le… Website: lean4agent.github.io/
🚀 Introducing the Lean4Agent framework. The first framework 🎉 that unifies the formal verification of agent workflow and executions. It views the agent workflow as “a computer program in natural language.” 🤖 The Lean4Agent provides FormalAgentLib that treats agent workflows like programs. We develop a predicate system that elegantly constrains the agent's execution step via preconditions and postconditions. ✅ This framework makes agent workflows’ self-consistency verifiable before execution, and enables the detection of which step of the workflow breaks the consistency for fine-grained fixes. 🔁 When a verified workflow fails on a specific problem, the predicate system, combined with the LeanEvolve proposed in the Lean4Agent framework. We can dynamically update the workflow to enhance its performance. [1/n] Paper: arxiv.org/abs/2606.06523 Huggingface: huggingface.co/papers/2606.0… GitHub Repo: github.com/RickySkywalker/Le…
2
17
4
31
118,412
1/7 Can an LLM learn from an older rollout model without importance weights? Introducing Gibbs-Gap Policy Optimization (GGPO). Sync rollouts every 8 -> 128 optimizer updates: accuracy stays at 46.58% -> 46.39% across five math benchmarks (Qwen3-1.7B-Base).
4
13
1
20
1,145
Jiarui Yao retweeted
1/11 Can a single-token importance weight preserve the target of full-sequence correction? With Future-KL, we prove that both updates share the same unique stationary policy. We also contribute the promising empirical results we observed. arxiv.org/abs/2610.18738
3
23
1
114
6,661
Jiarui Yao retweeted
🤔 AI agents can do a lot once you give them a task. Can they also figure out what’s worth doing and get started before being asked? 🏫 That’s the question behind Proactive AI for Superintelligent Agents, our joint survey by researchers from UIUC, Google, Stanford, UC Berkeley, and CMU. Our backgrounds span AI agents, embodied AI, information retrieval, computational biology, economics, and finance. 🕵 As finding papers gets cheaper, we think a survey’s value increasingly comes from hard-earned research intuition: knowing what matters and what to try next. Contributors drafted the sections closest to their own research, then refined them through several rounds of discussion and revision across research backgrounds. 📃 Paper: empathyang.github.io/Proacti… 🌎 Website: empathyang.github.io/Proacti… 💼 Google Research: research.google/pubs/proacti… More in the video below 👇
7
27
19
84
45,298
Jiarui Yao retweeted
We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. github.com/openai/math
1,903
6,306
4,768
40,715
22,732,479
Excited to present AdaPlanBench at #COLM2026! 🎉 We study whether LLM agents can continually revise plans as world and user constraints evolve, deciding what to preserve, what to revise, and how to remain consistent with accumulated requirements. 📍 Imperial Ballroom, Poster #22 🗓️ Tuesday, Oct. 6 🕟 4:30–6:30 PM PT Feel free to drop by and chat if you’re interested! I’m also 𝗮𝗰𝘁𝗶𝘃𝗲𝗹𝘆 𝗮𝗽𝗽𝗹𝘆𝗶𝗻𝗴 𝘁𝗼 𝗣𝗵𝗗 𝗽𝗿𝗼𝗴𝗿𝗮𝗺𝘀 𝗳𝗼𝗿 𝗙𝗮𝗹𝗹 𝟮𝟬𝟮𝟳. Would love to connect with potential advisors and labs working on LLM agents and related areas. Find me at my poster or DM me to chat! 🚀 📄 arxiv.org/abs/2606.05622
🎉 AdaPlanBench got accepted to COLM 2026. See you in San Francisco! 🤔 We ask a simple question: can LLM agents still plan reliably when constraints are hidden, feedback arrives progressively, and the world/user says “no” after the agent acts? 🔍 Our results show that current LLM agents still struggle with adaptive re-planning with world and users: they may forget previously revealed constraints, overreact to new feedback, or generate plans that appear valid but fail under accumulated constraints. Check out our paper: arxiv.org/abs/2606.05622 Big thanks to all mentors and collaborators, and special thanks to my advisor @hengjinlp for her guidance and support! P.S.: We are also continuing this direction with PlanBench-XL, a follow-up work focusing more on long-horizon adaptive planning with strategic exploration and exploitation in large-scale tool ecosystems. Check it out: planbench-xl.github.io/
12
30
1,289
Better agents may begin with better teachers. A capable PhD student can lose days to scattered experiment logs and broken workflows. A good advisor identifies these bottlenecks and builds the support that lets the student focus on scientific judgment. Our new work brings this idea to AI4AI: a Builder learns from a Target’s execution feedback to design better harnesses. We call these reusable lessons meta-skills. Both models’ weights stay frozen; what improves is how one AI supports another. Check out our newest paper: Learning Meta-Skills for Agent Harness Design in Test-Time AI4AI 📄 arxiv.org/abs/2609.38143 💻 github.com/qiancheng-apodex/…
3
21
2
78
5,392
🚀 CFP: 3rd Edition of the Lifelong Agent Workshop (LLA)! After ICLR 2026 and COLM 2026, we’re excited to bring LLA to AIAS+ 2026, back in San Francisco on Nov 7! We welcome work on all aspects of lifelong agents, including lifelong learning and adaptation, alignment and safety, agent evolution and self-improvement, evaluation and benchmarking, multi-agent systems, AI for Science, and applications across diverse domains. 📅 Submission deadline: Oct 10 (AoE) We’d love to see your latest ideas, insights, and ongoing work. Submit your work and join us in San Francisco! Website: lifelongagent.github.io/aias… OpenReview: openreview.net/group?id=aias… Registration: aiasplus.org/event-registrat… Hope to see you at LLA @ AIAS+ 2026! #LLMAgents #AIAgents #LifelongLearning #AIAS2026
2
16
1
32
2,243
Jiarui Yao retweeted
Building AI tutors requires feedback from students, but real learner studies are slow and hard to scale. 📈 Just in: StudentSim (Microsoft×UIUC) trains personalized AI student simulators 🤖 from real learner 🧑‍🎓 records and evaluates whether they both match a specific student’s behavior and can respond to tutor guidance. ✏️ Across three learning domains, StudentSim outperforms strong student simulator baselines, including GPT-5.4. In a chess tutor RL study, using StudentSim as guidance helpfulness feedback led to tutor guidance that human experts rated higher in accuracy, guidance quality, and personalization. 📄 Arxiv: 2609.01591 💻 Code: microsoft/StudentSim
4
43
1
166
20,393
Over the past year, we have worked on three projects around User-Centric Agents, and this line of work has finally come together at EMNLP this year. Behind all three projects, we have really been asking the same question: If the ultimate purpose of an Agent is to serve people, should we move beyond modeling tasks -- and start modeling users? Many Agent systems implicitly assume that users provide goals that are clear, complete, and stable. Real human interaction is almost the opposite: our needs may be underspecified, our preferences often emerge gradually through conversation, and our behavior is shaped by latent factors such as our beliefs, intentions, and perspectives. A truly user-centric Agent therefore needs to solve three increasingly deeper problems: 1. Can it understand and satisfy user needs through complex interaction? 2. Can it learn better interaction strategies by continuously engaging with users? 3. And ultimately, can it understand the unobservable mental states behind user behavior? 🍃🍃🍃 🧭 UserBench: From Completing Tasks to Understanding Users We first put the user back into the Agent's environment. In UserBench, user requests can be underspecified, incremental, and indirect. The Agent must actively communicate with the user, uncover preferences, and then use tools to accomplish the task. What we found is striking: even models that are already very strong at tool use often manage to "complete the task" without actually satisfying the user. This reveals an important distinction: --> Task success is not the same as user success. 🏋️ UserRL: From Evaluation to Learning The next natural question was: can these capabilities be learned systematically? UserRL builds 8 user-centric gyms, where Agents continuously interact with simulated users through multi-turn rollouts and learn via reinforcement learning to improve capabilities such as clarification, adaptation, persuasion, and intent understanding. Here, the user is no longer merely the provider of a prompt: --> The user becomes a dynamic, interactive training environment. 🧠 UserHarness: From Behavior to Mind But users are more than collections of preferences. They also have their own beliefs, intentions, and perspectives. UserHarness therefore takes one step further and reframes Theory of Mind as user-mind reconstruction: explicitly tracking what a user has observed, what they believe, what they intend to do, and how these internal states give rise to their behavior. --> The goal is no longer merely to predict what the user will do, but to understand why the user behaves that way. 🍃🍃🍃 Whether future Agents evolve toward AI4AI, RSI, or increasingly autonomous intelligence, one question will remain unavoidable: "Who is the Agent ultimately acting for?" Today's AI can already automate increasingly complex tasks. But the more autonomous these systems become, the more important it is that they genuinely understand human needs, preferences, intentions, and boundaries. The capabilities of future Agents may continue to expand, but their objectives, behavioral boundaries, and ultimate locus of control should remain firmly anchored to the user. Finally, hope to see many of you in Budapest this year! 📄 UserBench: arxiv.org/pdf/2507.22034 📄 UserRL: arxiv.org/pdf/2509.19736 📄 UserHarness: arxiv.org/pdf/2605.27721
1
14
61
2,761
When I first started doing research, I worked on a few projects around LLM confidence/reliability. The latter two were recently accepted to EMNLP, so I thought I’d wrap up this little line of work. 🔗 arxiv.org/abs/2505.24778 — Can LLMs reliably express confidence through linguistic markers? We found them surprisingly unstable under distribution shift. 🔗 arxiv.org/abs/2508.08992 — Can LLMs correctly interpret linguistic confidence and use it in decision making? Again, lots of instability. 🔗 arxiv.org/abs/2601.11004 — As factual tasks increasingly rely on retrieval, we studied how retrieval noise affects confidence calibration and how to improve it. Different settings, but one question: can models reliably express and perceive uncertainty, and reason and act accordingly? I think this is fundamental to reliable agents. Huge thanks to Prof. Song @yqsong and everyone I worked with! Confidence is truly a mysterious topic, with many nights spent staring at bizarre results 😂 Couldn’t have made it through all of this alone. This little line of work can finally take a pause :)
🚀 Introducing NAACL (Noise-AwAre Confidence CaLibration): a scalable, self-bootstrapping framework that trains LLMs to provide reliable and grounded verbal confidence in RAG scenarios.  📄 Paper: arxiv.org/abs/2601.11004 💻 Github: github.com/HKUST-KnowComp/NA… #LLM #RAG #Alignment
1
14
39
4,266
🚀 Flow-DPPO: Divergence Proximal Policy Optimization for Flow Matching Models We introduce Flow-DPPO, which replaces PPO-style ratio clipping with a divergence proximal constraint that is structurally inherent to flow models. 🔗arxiv.org/pdf/2606.11025
1
16
2
185
18,009
🚨 Uniform token-level trust regions are not enough for LLM RL! Our new paper: Beyond Uniform Token-Level Trust Region in LLM Reinforcement Learning. We introduce CPPO, a drop-in mask that reallocates divergence budget by position & prefix drift 👇 arxiv.org/pdf/2606.10968
3
28
5
179
37,387
Jiarui Yao retweeted
🚀 Introducing the Lean4Agent framework. The first framework 🎉 that unifies the formal verification of agent workflow and executions. It views the agent workflow as “a computer program in natural language.” 🤖 The Lean4Agent provides FormalAgentLib that treats agent workflows like programs. We develop a predicate system that elegantly constrains the agent's execution step via preconditions and postconditions. ✅ This framework makes agent workflows’ self-consistency verifiable before execution, and enables the detection of which step of the workflow breaks the consistency for fine-grained fixes. 🔁 When a verified workflow fails on a specific problem, the predicate system, combined with the LeanEvolve proposed in the Lean4Agent framework. We can dynamically update the workflow to enhance its performance. [1/n] Paper: arxiv.org/abs/2606.06523 Huggingface: huggingface.co/papers/2606.0… GitHub Repo: github.com/RickySkywalker/Le…
1
2
2
4
2,435
Jiarui Yao retweeted
Our latest deliveries: 👨‍💻 UniRL, an RL infra for unified multimodal models ➕ Two new RL algorithms: • FlowDPPO for diffusion and flow-matching models • DRPO for LLMs and VLMs Check out our repo: github.com/Tencent-Hunyuan/U…
🚀Introducing UniRL, an RL infra for unified multimodal models. Together with two new RL algorithms: DRPO and Flow-DPPO. One RL loop across diffusion/flow matching models, LLMs/VLMs, and unified multimodal models👇 Code: github.com/Tencent-Hunyuan/U… (yes — U(you)-ni-(need) RL 😉)
4
28
1
159
21,734
Jiarui Yao retweeted
We propose DRPO: a soft version of DPPO🔥 Since PPO, clipping/mask-based trust regions have long outperformed smooth divergence regularization like KL, even though the latter one feels more principled. 👺 We found two missing pieces:👇 1️⃣ Weight the regularizer by |advantage| - Otherwise, the trust region geometry changes dynamically and optimization becomes unstable. 2️⃣ Use the right divergence - What matters is not just “regularization”, but the trust-region geometry induced by the gradient. DPPO-style geometry works much better than PPO-style geometry in LLM. These insights lead to DRPO, which delivers the most robust and best overall performance across algorithms, even outperforming original mask-based DPPO. 🚀 This project is an amazing collaboration with @ExplainMiracles, @NickZhou523786, Wee Sun Lee, Liefeng Bo, @TianyuPang1 . Do follow them if you are interested in this work! 📄 Paper: arxiv.org/pdf/2606.09821 💻 Code: github.com/Tencent-Hunyuan/U…
3
27
2
155
12,546
✨ Creativity is not just recognizing what an object is — it is imagining what it could become. 🔧 A key edge can cut tape. 🛡️ A rubber pad can protect a wall. 🪮 A comb guard can clear a sink slot. But can multimodal AI agents discover these hidden physical affordances from images? 🚀We introduce MM-CreativityBench, a benchmark designed to test whether LMMs can creatively repurpose everyday objects by interactively inspecting scenes, entities, and object parts. 🔍 Our findings show that today’s LMMs often identify the right object, but fail to ground their reasoning in the right part. They hallucinate properties, overlook physical constraints, or propose solutions that are not mechanically valid. 🧠 To move beyond plausible guesses, we propose affordance-grounded alignment: training models to explore visual evidence, reject hallucinated affordances, and reason from geometry, material, and mechanics. 📄 Paper: arxiv.org/pdf/2605.26396 🌐 Project: creativitybench.github.io/mm… 💻 Code: github.com/CreativityBench/M… 🤗 Hugging Face: huggingface.co/papers/2605.2…
3
23
6
58
10,267
🚀 Wanna build your own customized agent with controllable workflow? Introducing AgentSPEX — a declarative DSL for building LLM agents. - Customizable agentic workflow with GUI builder - Reproducible State-of-the-Art in SWE-bench verified - Controllable workflow with YAML specification + sandboxed VM + Lean4 verification 🌐 Demo: agentspex.ai 💻 Code: github.com/ScaleML/AgentSPEX 📄 Paper: huggingface.co/papers/2604.1… 💡 (One of the) Applications: researchguide.work (1/n) #LLM #AIAgents #AgenticAI #OpenSource #AIResearch #AgentHarness
1
8
2
18
13,296