The Kim Jaechul Graduate School of AI at KAIST
Seoul, Republic of Korea
Joined March 2022
- Tweets199
- Following262
- Followers2.7K
- Likes188
Pinned Tweet
๐ NVIDIA ร KAIST Joint AI Research Lab is here!
๐ $300M for Sovereign Agentic Future
Announced by @JensenHuang himself in SF! The new lab at the Kim Jaechul Graduate School of AI brings top KAIST talent + @nvidia's full-stack tech + massive compute๐ฅ
nvidianews.nvidia.com/news/nโฆ
KAIST AI retweeted
๐ค Improving robot foundation models with real-world RL, then continually carrying that experience forward across tasks.
Introducing ๐๐๐๐ฅ๐จ๐: ๐ฐ๐ผ๐ป๐๐ถ๐ป๐๐ฎ๐น ๐ผ๐ณ๐ณ-๐ฝ๐ผ๐น๐ถ๐ฐ๐ ๐ฅ๐ for robot foundation models.
Our real-world policies learn through physical robot interaction, starting from a small set of demos.
We inherit the actorโcritic across tasks and distill earlier behavior, so experience accumulates while prior skills remain.
๐ป ๐๐ผ๐ ๐ฎ๐ฑ๐ฑ๐ถ๐๐ถ๐ผ๐ป๐ฎ๐น ๐๐ฃ๐จ ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด ๐ผ๐๐ฒ๐ฟ๐ต๐ฒ๐ฎ๐ฑ: RL updates a compact actorโcritic while the foundation model stays frozen.
โก Subsequent tasks reach 95% success in about ๐ฐโ๐ญ๐ฎ ๐บ๐ถ๐ป๐๐๐ฒ๐ ๐ผ๐ณ ๐ฟ๐ผ๐ฏ๐ผ๐ ๐ถ๐ป๐๐ฒ๐ฟ๐ฎ๐ฐ๐๐ถ๐ผ๐ป.
On real robots:
๐ Up to 68.5% less robot interaction vs. single-task RL
๐ฏ 98.3% final prior-task success
๐ฌ A 60-second overview below.
๐ accrue-rl.github.io/
๐งต 1/6
KAIST AI retweeted
Do language models always need to see the full 1M-token context window? What if they can just tell us which tokens they actually need?
We're excited to share Declarative Attention (DA), a sparse attention protocol that lets a model declare which context they need for each reasoning step, in plain text. The inference engine can use this to mask out up to 99% of KV cache reads for global attention.
Across 15 long-context tasks, DA reduces global attention KV reads by 46% with only 1.4pp accuracy drop on Gemma-4 31B and 20% with 2.1pp on Qwen-3.6 27B. This accuracy gap shrinks as we scale up the model size. Zero-shot.
We discuss further potential in agentic settings with dedicated post-training, and model-controlled KV cache offloading as a path toward reversible context compaction.
We explain details and link our paper below.
KAIST AI retweeted
Can explicitly learning future point motion improve the model that generates those future images?
We introduce VPM (Video-Point Model), a 4D world model that jointly forecasts future multi-view videos + persistent 3D point trajectories.
Short answer: yes. Teaching a video world model to forecast 3D point tracks makes its videos more faithful & 3D-consistent, and makes robots better.
๐ Project: rootyjeon.github.io/video-poโฆ
๐งต [1/6]
Can AI agents learn from their own failures? ๐ค
Excited to host Prof. @XiaoxiaoLi8 (@UBC, @VectorInst, @YaleMed) for an invited talk on how agents can diagnose failures, adapt, and recursively improve themselves, drawing lessons from vision and medical AI!
๐ค Made with AI
KAIST AI retweeted
๐ Agents are now taking over some of our knowledge work! But, after working through a stack of files over dozens of turns, something they saw slips out of what they deliver.
Excited to introduce RunningTab๐งพ, where the environment keeps a running tab of what the agent has read and what the task still owes, until the work is handed in.
From writing reports and checking budgets to pulling figures from a century of government records, it consistently outperforms the same agent without the tab and baselines that track the task in the model itself.
Paper: huggingface.co/papers/2610.1โฆ
KAIST AI retweeted
Introducing SQAM, Q-Learning with Scalar Adjoint Matching! ๐ Off-policy RL for flow policies, without a single VJP through the policy. Bigger gains on the hardest tasks, at a lower cost per update. ๐งต [1/8]
KAIST AI retweeted
Full-duplex dialogue models listen and speak at the same time, yet are still evaluated against a partner that cannot react.
โจ Meet DyaFDB: two full-duplex models converse directly, and we score both.
๐ Paper: arxiv.org/abs/2610.08125
๐ Page: dyafdb.github.io/
1/ ๐งต
KAIST AI retweeted
Speech+RL paper from @Qualcomm AI Research and @KAIST_AI !
For more natural conversational AI, we usually use full-duplex speech language models, which can perform smooth turn-taking and also produce uh-huh-like backchannels.
However, there is no clear answer to when a model should barge in. So in this field, we explore reinforcement learning rather than relying solely on supervised fine-tuning. But due to the nature of full-duplex SLMs, their token distribution is totally different from that of normal language models. So our idea is to build a reinforcement learning method for full-duplex speech language models. Our method, HiPLEX (Hierarchical policy factorization for full-duPLEX SLMs), factorizes the policy into when to talk and what to talk. Plus, our credit assignment method causally attributes rewards to the correct or incorrect timing and semantic decisions. (See Figure 1 featuring the cute San character!)
arxiv: arxiv.org/abs/2610.07727
huggingface paper: huggingface.co/papers/2610.0โฆ
Huge thanks to my Qualcomm speech team.
KAIST AI retweeted
Speech+Agent paper from @Qualcomm and @KAIST_AI
When we handle a cascaded speech agent system (ASR+LLM+TTS) for any reason, the LLM can call tools, but latency is a problem. We researched how we can hide the tool-calling latency and found that we can hide it within ASR time.
We tested it on the Samsung Galaxy S26 Ultra and confirmed that we can reduce latency and also (which we didnโt anticipate) improve performance!
Check out our paper!
arxiv: arxiv.org/abs/2610.07641
๐ค : huggingface.co/papers/2610.0โฆ
LinkedIn: lnkd.in/p/gSvihjfE
KAIST AI retweeted
Must-read papers of the week
Top:
โช๏ธ Context Language Models by @Meta
โช๏ธ Invent a Dataset by @adaption_ai
โช๏ธ World Observer by @KAIST_AI
โช๏ธ The Planning Limits of Latent World Models by @uommedia
โช๏ธ JEPA-TTT by @hri_usa and @JohnsHopkins
โช๏ธ False Frontiers by @RutgersU and co
Worth checking:
โช๏ธ EVO-WAM
โช๏ธ InterEvolve
โช๏ธ OneStreamer
โช๏ธ LoopVL: Recurrent Visual Intelligence
โช๏ธ WorldPlay2
โช๏ธ Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States
โช๏ธ LLMs are General Asynchronous Agents
โช๏ธ ActiveSaddler
โช๏ธ EVOKE
โช๏ธ GraphForge
Here is the full list of the most interesting papers + links and our weekly AI news digest:
turingpost.com/p/reflection-โฆ
KAIST AI retweeted
World models have one eye: the camera.
They can evolve what they see, but the rest of the world quickly goes unobserved.
So what if we gave them more eyes, like LoL wards or StarCraft Observers, and let them move around the world?
We introduce World Observer!
TL;DR: Keep the off-screen world evolving. Joint actor-observer generation with freely placed and movable panoramic observers.
Proj page: cvlab-kaist.github.io/world-โฆ
@KAIST_AI @hyunwookee
KAIST AI retweeted
Your emails, chats, docs, and tickets keep piling up, and the right ones get harder to find. ๐ก
๐บ๏ธ Introducing CorpusMap, which organizes a growing corpus around recurring entities, so agents follow connections instead of searching from scratch.
๐ Better answers, fewer tokens.
KAIST AI retweeted
๐ Excited to share that ES-Merging has been accepted to NeurIPS 2026! ๐
๐ก Biology connects molecules, proteins, and cells.
How can we bring their AI experts together?
We use embedding-space signals to merge specialized biological MLLMs into one model:
๐งฉ Combine expertise at both layer and parameter levels
๐ Improve cross-modal reasoning
๐ง Better preserve single-modal knowledge
Huge thanks to my co-authors @WonbinLee067 and @SungJuHwang1 for their collaboration and support!
Check out the paper and code for more details and results๐
๐ Paper: arxiv.org/abs/2603.14405
๐ป Code: github.com/WonbinLee067/es-mโฆ
@KAIST_AI @MLAI_KAIST
KAIST AI retweeted
๐ก Success under uncertainty is hard to repeat, while confident failures tend to recur. How can we learn from both to improve exploration in LLM reasoning?
Introducing EAPO (Entropic Advantage Policy Optimization), entropy-guided credit assignment for RLVR that treats success and failure asymmetrically to promote effective exploration in LLM reasoning:
๐ฏ Reinforce surprising success at high-entropy decisions
๐ง Penalize repeated failure more strongly at low-entropy decisions
๐งญ Soften penalties at uncertain decisions to preserve alternatives for recovery
Without auxiliary models, token-level supervision, or substantial additional computation, EAPO achieves the best overall results across both base and reasoning backbones on diverse reasoning benchmarks.
KAIST AI retweeted
Target representation matter for denoising models (e.g., RAE).
But what about the Source?
Can we learn source for better generation?
Flow matching doesn't require Source to be Gaussian noise.
Meet CSFM: end-to-end source learning for T2I.
๐ NeurIPS 2026
arxiv.org/abs/2602.05951
KAIST AI retweeted
New preprint from @KAIST_AI with an undergrad intern in our DAVIAN Lab.
We updated just the TTS acoustic model, and a deepfake detector had a harder time catching its speech. Show the detector the new outputs, though, and it learns to spot them again. Missed โ undetectable.
KAIST AI retweeted
New preprint from @KAIST_AI and @NAVER__Cloud
More audio tokens != better speech understanding.
At 512 dimensions, dropping our codec from 25 Hz to 12.5 Hz halved the token count and improved spoken QA from 29.7% to 34.6%. Sometimes less really is more.
"Not only does South Korea have the second-highest talent retention rate of 77 percent, its key institution KAIST leapt twelve places since 2022 to now rank as the third-highest institution in terms of producing top AI talent."
So proud of our entire KAIST familyโour students, faculty, alumni, and everyone who has helped build this community. Let's keep pushing! ๐
Full article from Carnegie: carnegieendowment.org/featurโฆ
Great idea! Interested in joining the discussion ๐
I propose Stanford NLP as an independent third-party evaluator under @DarioAmodeiโs 3 step plan. For important parts of the work, universities would be better than any other organization (see below ๐งต๐), and, of university groups, @stanfordnlp would be the best one to choose. ๐
