PhD Student @unccs working on AI Safety
United States
Joined July 2025
- Tweets29
- Following67
- Followers28
- Likes78
Pinned Tweet
Excited to share our #NeurIPS2026 paper "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"! 🎉
We show that LLMs contain latent safety signals in their first-token probability distributions and these signals can transfer across different models.
🎉 Excited to share our #NeurIPS2026 paper, "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"!
Latent Safety Signals in LLMs: Can we detect a harmful query before the model generates even one token? We show that the answer lies in the model's dark knowledge, i.e., the first-token probability distribution contains safety signals that transfer across different LLMs.
We use this insight to build LADE (Latent Safety Signals for Defense), a model-agnostic defense against jailbreak attacks that requires no access to internal hidden states. 🛡️
Key takeaways:
1️⃣ We discover latent safety signals in the first-token probability distribution: contrasting harmful and benign queries reveals safety-discriminative tokens that transfer consistently across safety-aligned LLMs, despite different architectures, tokenizers, and refusal styles.
2️⃣ We introduce LADE 🔍: a lightweight framework that identifies harmful queries before generation begins by applying a simple kNN classifier to latent safety signals in the first-token probability distribution.
3️⃣ LADE also transfers across models 🔄. Across all reference–target LLM pairs, it achieves 96.6% average harmful-vs-benign classification accuracy. It also performs well against several jailbreak attacks while keeping over-refusal low.
Wonjun Lee retweeted
🚨 Excited to share WEDGE: From Failures to Learnable Worlds, our new work on policy-conditioned environment generation for navigation agents!
We found that even on the same navigation task, agents tend to fail under different environmental conditions. Yet most environment generators do not consider what each agent actually needs to improve.
🤔 This led us to a question: Can we use an agent's own failures to guide the generation of training environments?
➡️ We introduce WEDGE, a framework where an agent's failures shape the worlds it trains in next. WEDGE identifies the conditions under which each agent struggles and generates new environments that target those weaknesses while remaining learnable. After each round of adaptation, it re-diagnoses the agent to guide the next round of environment generation.
Thread🧵👇
Wonjun Lee retweeted
🚨 As agents become more capable, new safety and security risks continue to emerge, as highlighted by recent reports of AI agents taking unauthorized actions or leaking sensitive data. Therefore, we need runtime guardrails that can reliably enforce safety policies before agent actions are executed, while continually adapting to new risks.
Existing guardrails usually trade off adaptability and reliability: LLM-based guardrails can adapt with memory, but their decisions may be inconsistent. On the other hand, formal guardrails provide deterministic enforcement but are typically fixed and unable to incorporate new risks that appear after deployment.
🚀 We introduce VEPA, a lifelong formal guardrail for agents. It represents safety and security policies as executable symbolic rules that evolve from deployment feedback. These policies enable deterministic enforcement of agent actions and can be formally verified, avoiding the inconsistency and ambiguity that can occur with natural-language policies.
🧵⬇️
Wonjun Lee retweeted
🚨 We are hiring Tenure-Track and Full Professors in important and timely areas such as AI Safety**, Neuroengineering, as part of exciting campus-level strategic cluster hires with CS, and with several positions in the clusters. The Full Professor position in 'Safeguarding AI for Health' is jointly between our strong BioStats department and CS -- please spread the word and feel free to apply & reach out for any questions!
**broad emphasis incl. but not limited to machine learning theory, AI systems, interpretability, robustness, verification, human-AI interaction.
We are hiring at @unccs 🎉
▶️ Two TT Asst. Professors: 1 in Safeguarding AI in Health, 1 in Neuroengineering.
▶️ Full Professor jointly with Biostats.
Part of campus cluster hires (review begins Dec 1).
Plus, Teaching Faculty (review begins Nov 1).
cs.unc.edu/about/faculty-hir…
Wonjun Lee retweeted
Can agents learn useful skills from frozen logs that people and agents already leave behind?
🚨Introducing TeleTune: We turn noisy offline telemetry into agent skills w/o access to a live environment during training!
+6.7% on WorkArena & +7.7% on Mind2Web 👉 up to 75x fewer tokens used for optimization. 🧵
Activity logs like user telemetry capture how people use software, but often (1) lack explicit task goals, (2) cannot be replayed to test each skill update, and (3) can be a noisy mix of multiple tasks.
We address these jointly in TeleTune, where we propose skill-guided progress, an offline metric that uses action-prediction errors to improve agent skills. Not only does optimizing it improve downstream performance, but it also saves token usage by 5 to 75x compared to validating skill updates in live environments!
Key takeaways:
1️⃣ TeleTune achieves 77.1% average success on WorkArena and 80.6% on Online-Mind2Web, outperforming the strongest baselines by +6.7% and +7.7% absolute
2️⃣ Optimizing offline skill-guided progress transfers to improvements in downstream task success
3️⃣ Offline optimization uses 5 to 75x fewer tokens than validating the skill edits in a live environment
Wonjun Lee retweeted
Excited to share WOVEN! 🚀
The goal isn't just to teach models what the world looks like. It's to teach them how the world changes.
Our key finding: Visual transition reasoning is a shared, teachable primitive that transfers across diverse downstream tasks.
And the key to effective transfer? Teach the right reasoning operations, rather than simply matching scenes, actions, or domains.
🚨 What makes good world-model training for MLLMs that actually transfers across tasks?
🧶 Excited to share our new work, WOVEN!
🧩 Spatial, physical, temporal, and embodied reasoning all require understanding how the world changes.
🔁 We view world modeling through state transitions p(s′ | s, a), and study visual transition reasoning: inferring what is unobserved in (s, a, s′).
🤔 Can visual transition reasoning be learned as a shared primitive that benefits multiple downstream tasks?
🤔 If so, how should we train it for better downstream gains?
Major takeaways:
➡️ Visual transition reasoning is a shared, teachable primitive that transfers widely downstream: ~2K-example subsets collectively improve 22 of 26 downstream benchmarks by up to 27.3 points.
🧪 A data-centric training recipe: to teach this shared primitive for downstream tasks, select supervision by the reasoning operation it teaches, rather than scene/action/domain similarity.
Thread🧵👇
Wonjun Lee retweeted
🎉 Excited to share our #NeurIPS2026 paper, "Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge"!
Latent Safety Signals in LLMs: Can we detect a harmful query before the model generates even one token? We show that the answer lies in the model's dark knowledge, i.e., the first-token probability distribution contains safety signals that transfer across different LLMs.
We use this insight to build LADE (Latent Safety Signals for Defense), a model-agnostic defense against jailbreak attacks that requires no access to internal hidden states. 🛡️
Key takeaways:
1️⃣ We discover latent safety signals in the first-token probability distribution: contrasting harmful and benign queries reveals safety-discriminative tokens that transfer consistently across safety-aligned LLMs, despite different architectures, tokenizers, and refusal styles.
2️⃣ We introduce LADE 🔍: a lightweight framework that identifies harmful queries before generation begins by applying a simple kNN classifier to latent safety signals in the first-token probability distribution.
3️⃣ LADE also transfers across models 🔄. Across all reference–target LLM pairs, it achieves 96.6% average harmful-vs-benign classification accuracy. It also performs well against several jailbreak attacks while keeping over-refusal low.
Wonjun Lee retweeted
Hybrid LMs (e.g., Qwen3.5, Nemotron-H) have both attention and recurrent memory. But do they actually learn to use both?
We find that even after SFT, models still rely heavily on attention, leaving recurrence underused.
How can we help models make better use of this underused pathway?
💡We introduce a simple auxiliary loss that forces past information through recurrence during training. This improves performance on QA and agentic tasks, with larger gains on longer contexts!
It turns out having multiple memory pathways is only part of the solution. Models also need to learn to use them effectively!
More findings and analysis in the thread 🧵👇
Recurrent–attention hybrid LMs (e.g., Qwen3.5) have two complementary memory pathways: attention and recurrence. But do they use both effectively?
❌ We find that they rely heavily on attention, even after SFT.
💡 We show that encouraging recurrent memory use improves performance (esp. on long-context tasks)!
🌐 Strong generalization across:
- Multiple hybrid LMs
- Question answering (avg. 4.6% ⬆️) and agentic tasks (avg. 12.1% ⬆️)
- Attention-only models with multiple memory types (28.6% ⬆️)
Wonjun Lee retweeted
Recurrent–attention hybrid LMs (e.g., Qwen3.5) have two complementary memory pathways: attention and recurrence. But do they use both effectively?
❌ We find that they rely heavily on attention, even after SFT.
💡 We show that encouraging recurrent memory use improves performance (esp. on long-context tasks)!
🌐 Strong generalization across:
- Multiple hybrid LMs
- Question answering (avg. 4.6% ⬆️) and agentic tasks (avg. 12.1% ⬆️)
- Attention-only models with multiple memory types (28.6% ⬆️)
Wonjun Lee retweeted
Glad to announce that MetaCanvas has been accepted to #NeurIPS2026! ✨
MetaCanvas lets MLLMs reason and plan directly in spatial and spatiotemporal latent spaces, instead of reducing them to global text encoders for diffusion models. A set of learnable "canvas tokens" carries the MLLM's planning into the diffusion latent space through a lightweight connector, and visualizations show these canvas tokens act as reasonable visual planning sketches guiding the final synthesis.
Implemented on three diffusion backbones and evaluated across six tasks: text-to-image, text/image-to-video, image/video editing, and in-context video generation, each requiring precise layouts, robust attribute binding, and reasoning-intensive control. MetaCanvas consistently beats global-conditioning baselines, narrowing the gap between multimodal understanding and generation.
See you in Atlanta! 🚀
Multimodal LLMs (MLLMs) excel at reasoning, layout understanding, and planning—yet in diffusion-based generation, they are often reduced to simple multimodal encoders.
What if MLLMs could reason directly in latent space and guide diffusion generation with fine-grained, spatiotemporal control? 🤔
Introducing MetaCanvas 🎨
A lightweight framework that translates MLLM reasoning into structured spatiotemporal conditions for diffusion models.
🧵 👇
Wonjun Lee retweeted
Hello from SF! 🌉 Excited to be at #COLM2026 and present our work "Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression" (TRAAC).
Curious how attention-based compression and difficulty-aware budgets can fix under- and overthinking in reasoning models? Come check out our work!
📍 Oct. 6, 4:30 PM PT, Franciscan C, Poster #98 (Poster Session 2)
Looking forward to discussing and sharing thoughts on agents, RL, and uncertainty in LLMs. Happy to grab coffee too ☕️
Paper: arxiv.org/abs/2510.01581
Code: github.com/joykirat18/TRAAC
Awesome Collaboration with @cyjustinchen, @ArchikiPrasad, @EliasEskin, Akshay Nambi, @mohitban47
@unccs @unc_ai_group
🚨 Excited to announce TRAAC, an online difficulty-adaptive, attention-based method that handles the tradeoff of under & overthinking in reasoning models to improve both accuracy and efficiency.
Underthinking ❌: Models terminate reasoning too early on harder problems, leading to errors.
Overthinking 🔁: Models think excessively for simpler tasks, generating unnecessary steps and inflating test-time computation.
1️⃣: TRAAC dynamically allocates the “thinking budget” via: Difficulty Calibration – adapt to problem hardness and Attention-Based Compression – prune redundant steps.
2️⃣: Across a variety of tasks (AIME, AMC, GPQA-D, BBEH), TRAAC (Qwen3-4B) achieves an average absolute accuracy gain of 8.4% with a relative reduction in reasoning length of 36.8% compared to the base model.
3️⃣: Analysis shows task-difficulty calibration and attention-based compression are effective for both accuracy and efficiency gains.
Wonjun Lee retweeted
I’ll be at #COLM2026 this week presenting EgoMemReason!
Our benchmark asks how well multimodal agents can build and use memory to reason across a week of everyday life-tracking object changes, connecting past events, and inferring recurring patterns.
📍 Poster #71, Imperial Ballroom
🗓️ Tuesday, Oct. 6, 4:30–6:30 PM PDT
Stop by to chat about the work and what comes next: how can we build persistent memory that helps agents learn from experience over time?
Exciting work with Yue Zhang @zhan1624 (co-lead), Shoubin Yu @shoubin621, Ce Zhang @cezhhh, Zengqi Zhao, Jaehong Yoon @jaeh0ng_yoon, Hyunji Lee @hyunji_amy_lee, Gedas Bertasius @gberta227, and Mohit Bansal @mohitban47 (@unc_ai_group @unccs)!
🌐 Website + Benchmark: egomemreason.github.io/
📄 Paper: arxiv.org/abs/2605.09874
I’m also on the industry job market for opportunities starting summer 2027. Would love to connect!
🚨 Excited to share EgoMemReason, a benchmark for multi-level memory-driven reasoning (entity, event, and behavior memory) over week-long egocentric videos (average 25.9 hours of temporal backtracking)!
📉 Current long video approaches can retrieve isolated event, but struggle with long-horizon memory that requires retrieve and understand across multiple events and long time: tracking evolving entities across days, linking temporally distant events, and abstracting recurring behavior patterns from long observations.
🎥 EgoMemReason evaluates these challenges through 500 human-verified questions spanning entity, event, and behavior memory, requiring aggregation over an average of 5.1 evidence segments and 25.9 hours of temporal backtracking.
⭐️ Across 17 models/frameworks, even the best model achieves only 39.6% accuracy, revealing that long-horizon multimodal memory remains far from solved.
Wonjun Lee retweeted
It was a pleasure to give a keynote at the 36th IEEE International Workshop on Machine Learning for Signal Processing (#MLSP2026), in Atlanta! 🙂
In my talk, “AI Challenges: Trustworthy Collaboration, Skill+World Discovery, and Long-Horizon Memory,” I discussed important and timely AI challenges in trustworthy+safe collaboration (incl. some latest adversarial scenarios), world discovery, and long-term memory planning, across diverse domains & modalities & real-world applications (e.g., early medical diagnosis and classroom education engagement)!
Thanks again to the conference organizers (Sergey Plis, Vince Calhoun, Ran Xiao, Sundaresh Ram, Carl Yang, Hyeokhyen Kwon, Ph.D., Bradley T. Baker, TReNDS Center) for the kind invitation and great organization (incl. the aquarium gala dinner with the whales+sharks, and the cool soundwave clip gift idea perfectly fitting a signal processing conference 🙂), with a very engaging audience/attendees! 🙏
Wonjun Lee retweeted
Excited to share that StreamGaze 👓, developed in collaboration with Adobe Research, has been accepted to #NeurIPS2026 E&D track (with score 5/5/4)! 🎉
We introduce the first benchmark for evaluating how well MLLMs can use human gaze signals 👀 for temporal reasoning in streaming egocentric videos.
More details below 👇
🤔 We rely on gaze to guide our actions, but can current MLLMs truly understand it and infer our intentions?
Introducing StreamGaze 👀, the first benchmark that evaluates gaze-guided temporal reasoning (past, present, and future) and proactive understanding in streaming video settings.
➡️ Gaze-Guided Streaming Benchmark: 10 tasks spanning past, present, and proactive reasoning, from gaze-sequence matching to alerting when objects appear within the FOV area.
➡️ Gaze-Guided Streaming Data Construction Pipeline: We align egocentric videos with raw gaze trajectories using fixation extraction, region-specific visual prompting, and scanpath construction to generate spatio-temporally grounded QA pairs. This process is human-verified.
➡️ Comprehensive Evaluation of State-of-the-Art MLLMs: Across all gaze-conditioned streaming tasks, we highlight fundamental limits of current MLLMs. All MLLMs fall far below human performance. Models particularly struggle with temporal continuity, gaze grounding, and proactive prediction.
Wonjun Lee retweeted
🎉 Excited that our NSF AI Institute for Engaged Learning has been renewed for another 5 years 🙏
Looking forward to leading the core AI efforts on 3D world models for multimodal narrative environments and AI that adapts to student needs -- and collaborating with our wonderful partners (top experts in education and learning sciences) to produce transformative advances in our understanding of how students learn and fundamentally improve the learning outcomes for K-12 students!
🚨 We'll be hiring postdocs+ phd students etc. for the next 5 years for this, so apply+ping with any questions!
🎉 @NSF renewed the EngageAI Institute for 5 more years and $20M. UNC CS researchers, led by Mohit Bansal (@mohitban47) will run the institute's core AI research: the world modeling tech behind story-based learning tools for K-12 science classrooms. unc.edu/posts/2026/09/14/ai-…
@unc
Wonjun Lee retweeted
Happy to share that MINTEval is accepted to #NeurIPS2026! 🥳
How well do LLM agents and memory systems handle long-horizon, continually changing environments (e.g., Wikipedia pages, Git repos)? We find they struggle to track changes, recall precise details, and combine information across interfering updates, mainly due to limitations in retrieval and memory construction. Models confuse old and new information, memory systems tend to insert redundant entries, and retrieval often fails to recover the right information.
Details 👇
LLM agents & memory systems operate in continuously updated environments (Git repos, evolving docs). They must process long contexts, recover earlier information, and reason over many updates that create interference between old and new information. How well do they handle this?
We introduce MINTEval:
✅ Frequent context changes & interference (avg. 86 updates)
✅ 5 challenging question types, including long-range lookback & reasoning over multiple targets distributed across context
✅ 4 realistic domains: state tracking, multi-turn dialogue, Wikipedia revisions, GitHub commits
✅ Avg. 138.8k tokens per instance (up to 1.8M)
✅ Human verification on generated QAs = 95.6%
📊 Across 7 representative systems, MINTEval remains difficult, showing an avg. acc of 27.9%, and the best system reaches only 33.4%.
🔎 Our analysis shows:
• Memory construction failures cause a 41.7% drop
• Memory agents are highly sensitive to design choices
• Memory systems have a strong bias toward insertion operations (76.8%) over deletion/update
Wonjun Lee retweeted
Happy to share that AVSD is accepted to #NeurIPS2026! 🎉
We study how to get better on-policy token-level learning signals for LLM reasoning. Self-distillation provides dense token-level supervision by matching a teacher policy conditioned on privileged information, but what if we could use multiple views of privileged information (hints, partial/full solutions, execution outputs, etc.), each inducing a different teacher distribution?
Our key idea is that useful supervision comes from both what the teachers agree on and information unique to individual teachers. AVSD adaptively combines both to produce better token-level learning signals for math and code reasoning.
🧵⬇️
Sparse binary rewards bottleneck LLM RL, motivating the use of privileged information in self-distillation as dense teachers. How can we use and balance multiple types of privileged info: leveraging stable cross-view info, while preserving view-specific info?
Current on-policy self-distillation methods often condition the teacher on only one type of privileged view: full solution, partial rationale, answer-only, reference code, feedback, etc. This can be suboptimal:
1️⃣ No single privileged view consistently performs best when used as a teacher.
2️⃣ Views can introduce teacher-specific artifacts from information unavailable to the student.
🧠 Adaptive-View Self-Distillation (AVSD) considers multiple privileged views jointly as a teacher family, balancing cross-view consensus and view-specific signals through a token-level gate to construct better dense learning signals.
🧵👇
Wonjun Lee retweeted
🎉 Excited to announce that AVIC has been accepted to #NeurIPS2026!
More imagination is not always better for spatial reasoning, the key is knowing when to imagine and how much to imagine.
AVIC learns an RL policy that adaptively invokes a video world model and controls how much to imagine, balancing answer accuracy against imagination cost for more accurate and efficient reasoning.
We're also grateful for the AC's metareview comment "I strongly recommend accepting this paper as a Spotlight" 🙏
More details below 👇
🚨We updated Adaptive Visual Imagination Control (AVIC) with a new RL extension: AVIC-R.
In AVIC, we proposed adaptive visual imagination for spatial reasoning and found that world models are most useful when invoked selectively.
In AVIC-R, we take this one step further by training a policy with RL to decide when to imagine and how much to imagine. Using GRPO with QA correctness and imagination cost as rewards, AVIC-R learns to use world models more efficiently and improves spatial reasoning without relying on fixed test-time scaling.
Notably, a Qwen2.5-VL-7B policy trained with AVIC-R outperforms GPT-4o / GPT-4 as policy models, showing that RL can teach smaller models to make better adaptive imagination decisions.
See more details in Arxiv v2 👉arxiv.org/abs/2602.08236
(original thread below 👇)
Wonjun Lee retweeted
Thrilled to announce that PhyMotion has been accepted to #NeurIPS2026! 🎉
PhyMotion brings 3D physics into the reward loop for human video generation. Instead of judging motion only from pixels or VLM features, we reconstruct human motion in 3D and evaluate its kinematic plausibility, contact & balance consistency, and dynamic feasibility through physics-based simulation. Using PhyMotion for RL post-training substantially improves the physical realism of generated human motion.
For the updated version, we also added extensive new analyses on robustness to 3D reconstruction errors and severe body distortions, terrain-aware evaluation on stairs, potential RL reward hacking/motion suppression, and challenging qualitative cases including multi-person interaction, occlusion, camera motion, and non-flat terrain.
Updated paper + code/model/data + lots of new video demos coming soon.
See you in Atlanta! 🚀
Details 👇
phy-motion.github.io/
🚨 Excited to introduce PhyMotion🤸: Structured 3D Motion Reward for Physics-Grounded Human Video Generation!
❌ Existing 2D video rewards misleadingly assign high scores to videos with floating feet, self-penetrating limbs, and physics-violating motions.
✅ PhyMotion lifts generated videos into 3D, grounds them in a physics simulator, and scores motion along kinematic / contact / dynamic feasibility.
➡️ RL post-training with PhyMotion improves 1.3B model to match 14B models performance in human prefence.
🧵(1/n)👇
Wonjun Lee retweeted
Honored to receive the Outstanding Faculty Mentor Award from the UNC Office of Postdoctoral Affairs, Provost, and Vice Chancellor for Research! 🙏
This one is especially meaningful because mentoring is very close to my heart -- I strongly believe it is one of the main reasons we are in this job, and it is our humble contribution in giving back to the community and helping create the next generation of mentors (students/postdocs/junior faculty, not just in our own lab/department but more broadly across other universities+countries) -- thanks to all my amazing mentees (it's a pleasure and privilege to mentor you) 🤗 💙
