
Assistant Professor @siebelschool UIUC. Researcher on Software Engineering & AI.
Joined June 2016
- Tweets125
- Following345
- Followers1.5K
- Likes485
Yiling Lou retweeted
Excited to share that our paper on re-evaluating LLMs for equivalent mutant detection received the ACM SIGSOFT Distinguished Paper Award at ISSTA 2026! 🎉🏆 Huge thanks to our amazing student interns and my wonderful collaborators, Darko and @yiling__LOU ! So grateful for this fantastic team!
📄 Check out our paper: wenxiwang.github.io/papers/L…
🚨 New work: Are we testing the agent harness enough?
Introducing HarnessTester: a harness-oriented test generator for LLM agents. It builds contract-faithful tests to exercise LLM-dependent harness code, and aims for exposing the bugs in the interaction boundary between LLM and harness. HarnessTester has detected over one hundred real-world harness bugs across widely-used agent systems (e.g., OpenClaw). (1/n)
📖: arxiv.org/pdf/2610.04921
More importantly: does HarnessTester uncover real harness bugs? On latest release of studied agents, HarnessTester finds 122 bugs in widely-used agentic systems (e.g., OpenClaw, Kimi Code), with 88 previously unknown and 69 already confirmed by developers. (5/n)
The takeaway: reliable agents require more than reliable models. As the harness becomes a major part of agent capability and complexity, we need systematic testing techniques designed specifically for this software layer. (6/n)
📄 arxiv.org/abs/2610.04921
Yiling Lou retweeted
Self-improving AI agents will need to fix their own code.
And this paper from top US+China labs, shows coding agents miss most such bugs but improve with lessons from past fixes.
that real bugs in agent harnesses, can be automatically turned into a growing set of runnable tests.
An agent's own code is everything around the model: tool calls, memory, and prompts. Its bugs depend on live model calls, which makes them hard to recreate and test.
So the researchers built AgentBug-Smith, which turns real GitHub bug reports into runnable tests. The result is a 200-bug benchmark that keeps growing.
The best of 3 coding agents fixed just 9% of those bugs, versus about 40% reported on regular software bugs. A short guide of lessons from past fixes lifted an agent from 1 to 6 correct fixes on 79 unseen bugs.
Before trusting a coding agent with your agent's code, try it on bugs you've already fixed.
Yiling Lou retweeted
Thanks for the shoutout to TLA+ @bcherny !
As suggested by Markus Kuppe, let me showcase the brilliant work done by colleagues at Specula and the TLA+ community. While Specula (github.com/specula-org) can effectively model system code in TLA+ (and then use model checkers to find hundreds of deep bugs), we can further prove the correctness of system specs using TLAPS. The TLAPS-Bench (github.com/specula-org/TLAPS…) project aims to offer a proof service for any TLA+ spec of important systems/protocols. Basically, Specula writes the spec; TLAS-Bench generates the proof.
AI writes amazing proofs! We had to retire all Proof-Completion tasks in TLAPS-Bench as they're too easy for frontier AI. We also proved classic protocols and algorithms like 2PC, Paxos, TCP, etc.
However, proving low-level specs of real-world system code _from scratch_ is still non-trivial. @qiancheng9788 ran a subset of TLAPS-Bench using Claude Code w/ Opus 5. The success rate is merely 51.4%. @Muse is great, but Muse Spark 1.3 can only achieve 16.7%.
Ruize has a bag of tricks to push AI agents. Once he pushed Codex to prove one of the nine invariants for the ZooKeeper implementation. Codex struggled 5 days, burned $2000, and did it correctly. We ran out of money to continue. And, @xu_dong_sun shows us that liveness proofs are still difficult for AI without guidance. We have an exciting avenue, but still a lot of work to be done.
If anyone wants to verify their protocols/systems, send us some tokens -- we can do it in TLAPS-Bench. @HacksonClark tells me that people are excited about long-horizon tasks -- TLAPS-Bench may be a perfect benchmark for that.
@bcherny Consider further tuning Claude for TLA+, you may find more bugs and/or prove the correctness of SDK code more efficiently.
I used Opus 5.5 to formally verify the Claude Agent SDK using Lean. A couple short prompts = 16 PRs fixing various bugs and race conditions. Video attached.
TLA+ also works well. I sometimes combine Lean and TLA+ to look for issues around data flow, concurrency, and state mgmt.
I don't know either language well, but Claude is excellent at both. This approach is super useful for formally modeling your code and finding bugs that a human probably wouldn't have spotted.
Is formal verification the future of coding (or at least, bug finding)?
Yiling Lou retweeted
astra as our first cyber critical model that 100%'ed public ExploitBench.
we made a contamination-free, internal port based bugs after astra's knowledge cutoff, where astra is 4x more cyber capable than 5.6 at fewer tokens. yet, team cooked hard to make astra v aligned.
As we prepare to release Astra, we’re focused on making increasingly capable AI safe and broadly accessible.
Astra represents a significant advance in cybersecurity capability, reaching the Critical threshold under our Preparedness Framework.
We're previewing how we evaluated the model, how its safeguards have advanced alongside its capabilities, and what we'll continue to learn and improve.
openai.com/index/path-to-ast…
Yiling Lou retweeted
Coding models are running out of data 🧵
The ability of LLMs to code depends heavily on the data they see about code, and lately the industry seems to have consumed essentially all high-quality code data for training and evaluation. Progress is visibly slowing. Whoever finds the next big pool of data holds the key to a new door.
My group came from programming languages and formal methods, and we know where such a pool is. More than that, we believe this data changes what an AI answer is: code should ship with its specification and a machine-checked proof, so we no longer need to spend the human time to review the code itself.
We call this new paradigm intent computing, and we ran several experiments...
🚀 Recruiting students for UIUC++ summer research in LLM4Code, software–AI synergy, software dependability …
Work on cutting-edge SE + AI/Robotics research with us. 📅 Apply by April 15
🔗 forms.gle/BUS8e3RfG946rEp68 @LingmingZhang @yiling__LOU @tianyin_xu @tan_hwei @DrWingLam
If you are excited about software engineering research, apply to our 2026 UIUC++ Summer Research Program! This is a fantastic opportunity to work on exciting problems in SE with amazing faculty! Many past interns have got into top 10 PhD schools!
forms.gle/BUS8e3RfG946rEp68
RT!
Yiling Lou retweeted
🤯🤯🤯 Gemini 3 Pro + Live-SWE-agent hits 77.4% on SWE-bench Verified, beating ALL existing models, including Claude 4.5!!
🤖 Live-SWE-agent is the first live software agent that autonomously self-evolves on the fly — and it even outperforms the manually engineered scaffold used by the Gemini 3 Pro team (76.2%)
Yiling Lou retweeted
Work with Xudong! Absolutely brilliant researcher and great person. Xudong will be an outstanding advisor!
I'm recruiting PhD students at the University of Toronto ECE. Our group aims to develop practical verification techniques for real-world, large-scale cloud systems. Apply to @eceuoft if you are interested in cloud systems, formal verification, or software testing.
Yiling Lou retweeted
⏰ 3 days left to submit to #LLM4Code2026 (co-located with @ICSEconf 2026)!
Pls submit by Oct 31: llm4code.github.io
We hope to create a platform for researchers/practitioners to discuss the latest in LLMs/Agents for Code across diverse fields: SE, PL, ML, and beyond.
Excited to announce the 3rd LLM4Code workshop collocated with ICSE26 @ICSEconf in Rio, Brazil🇧🇷!
🌟Website: llm4code.github.io
🌟Deadline: Oct 31
🌟8-page research paper/4-page position paper (including references)
@LingmingZhang @yiling__LOU @devanbu @Kexin_Pei @llm4code
Yiling Lou retweeted
Excited to announce the 3rd LLM4Code workshop collocated with ICSE26 @ICSEconf in Rio, Brazil🇧🇷!
🌟Website: llm4code.github.io
🌟Deadline: Oct 31
🌟8-page research paper/4-page position paper (including references)
@LingmingZhang @yiling__LOU @devanbu @Kexin_Pei @llm4code
Yiling Lou retweeted
🌟🌟🌟We are thrilled to announce AIware 2025 keynote speakers! For more details, please visit AIware website: lnkd.in/eNpDwRcW
📅Early bird registration deadline: Oct 15th
Registration link: lnkd.in/eNgzvz_x
Thrilled to announce that I'll be joining UIUC CS @siebelschool as an Assistant Professor in Spring 2026!
📢 I’m looking for Fall '26 PhD students who are interested in the intersection of Software Engineering and AI, especially in LLM4Code and Code Agents. Please drop me an email if you are interested in working with me.
Yiling Lou retweeted
📢 Excited to announce #LLM4Code 2026 (co-located with @ICSEconf 2026)!
🎯 Pls submit by Oct 20: llm4code.github.io
We hope to create a platform for researchers/practitioners to discuss the latest in LLMs/Agents for Code across diverse fields: SE, PL, ML, and beyond.
Announcing the 3rd workshop on #LLM4Code, co-located with @ICSEConf 2026 in Rio de Janeiro, Brazil 🇧🇷!
🎯We are calling for submissions:
🌟Website: llm4code.github.io
🌟Deadline: Oct 20, 2025
🌟8-page research paper / 4-page position paper (including references)
🚨 10 days left to submit to AIware 2025!
🏆🏆The best full-length papers will be recognized with an ACM SIGSOFT Distinguished Paper Awards
📢 Call for Papers
We invite submissions on the future of software in the FM era, including AI-powered development practices, LLMOps, trust, and SE education.
📄 Submission types: full papers, short papers, and literature reviews.
🔗 Full CFP: 2025.aiwareconf.org/track/ai…

























