Assistant Professor @ The University of Hong Kong, Previously Research Scientist @ DeepMind
Hong Kong
Joined February 2010
- Tweets116
- Following308
- Followers1.2K
- Likes133
Lingpeng Kong retweeted
Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved?
Not even close.
🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks.
What's new:
🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human
⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0
🌍 Grounded in authentic artifacts & stateful user profiles
⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more
📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use.
🏠 Homepage: osworld-v2.xlang.ai
📄 Paper: github.com/xlang-ai/OSWorld-…
💻 Code: github.com/xlang-ai/OSWorld-…
🤗 Dataset: huggingface.co/datasets/xlan…
🧵 [1/8]
Lingpeng Kong retweeted
Our work was cited by π0.7! This further highlights an important direction for VLA training: leveraging abundant human video data to address the bottleneck of task diversity.
Excited to share our new paper: CLAP: Contrastive Latent Action Pretraining for Learning Vision-Language-Action Models from Human Videos.
CLAP aligns human video dynamics with executable robot actions, enabling VLAs to transfer manipulation skills from human demonstrations to real robot control.
We will open-source the code in May.
arXiv link: arxiv.org/pdf/2601.04061
video on youtube: youtube.com/watch?v=oUGQQcFn…
Great to see the AI applications from local teams in HKU!
🚀 Turn ANY investment idea or research article into a backtested quant strategy. Let's have a try!
🌐 Pro Beta Investment Assistant Web App (🎁Sign up now to claim 500 FREE credits!): alagent.cloud/
🔗 Lightweight Open-Source Agent Skill: github.com/ALAGENT-HKU/x2str…
Lingpeng Kong retweeted
Agents are doing real work, but existing benchmarks still test them in isolation.
Today we’re releasing Claw-Eval 🦞: an open-source, transparent evaluation framework for AI agents.
We feature 104 tasks spanning daily assistants, Office QA, deep finance research, and terminal usage.
We test completion, robustness, and safety across real and mock services with configurable error injection.
Fully traceable and human-verified.
First leaderboard results: Claude Opus 4.6 @AnthropicAI tops pass rate (68.3%), but Gemini 3.1 @GeminiApp Pro edges it on avg score (0.764 vs 0.759).
Agents have a long way to go.🤨
Check it out: claw-eval.github.io
@steipete @openclaw
Lingpeng Kong retweeted
Introducing proxy compression for end-to-end language modeling: train on compressed (e.g., tokenized) data for efficiency, but run inference entirely on raw bytes without a tokenizer. No architectural changes required. At scale, proxy-trained byte models match or surpass tokenizer baselines at 7B and 14B.
📄 Paper: arxiv.org/abs/2602.04289
💻 Code: github.com/LZhengisme/proxy-…
[1/9]
🧵👇
🚀 Introducing Dream-VL & Dream-VLA!
We’re proving that dLLMs have an amazing advantage in building VLA models.
The result is stunning performance:
🏆 97.2% on LIBERO
⚡ 27x speedup vs AR models
🔥 Beats OpenVLA & $\pi_0$
✅ Fully Open Source
Blog: hkunlp.github.io/blog/2025/d…
Smarter TTS by Xueliang! See everyone in this year NeurIPS!
🚀 Thrilled to share our #NeurIPS2025 paper
DynaAct: Large Language Model Reasoning with Dynamic Action Spaces
A new test-time scaling view — optimizing the action space itself, while providing a general MCTS acceleration framework for reasoning.
💻 github.com/zhaoxlpku/DynaAct
Lingpeng Kong retweeted
This is super cool. I strongly believe that the flexibility of dLLMs during generation will enable new features for agent use.
Diffusion LLM + Agents are 🔥
This is @_inception_ai's Diffusion LLM with @huggingface SmolAgents:
- Planning tool use
- Executing 20 web searches and parsing results
- Synthesizing the data
All in 3.5 seconds. With 10 searches it took only 1.6 seconds.
Source on GitHub below.
Lingpeng Kong retweeted
We will have a guest talk from Cai Zhou. He is a second-year PhD in MIT EECS. "Continuous modeling in diffusion language models: HDLM and CCDD
". All are welcome to join via the following link.
hku.zoom.us/j/91579619608?pw…
Lingpeng Kong retweeted
DeepSeek-OCR: Exploring the boundaries of visual-text compression.
Ambitious! They might use 10X (near-lossless) compressed vision tokens to replace the KV cache of dialog histories.
github.com/deepseek-ai/DeepS…
Lingpeng Kong retweeted
The full Dream-Coder pipeline is now open-sourced—covering data prep, training, and evaluation. Check it out! github.com/DreamLM/Dream-Cod…
Saw the paper like a month ago. Now with the demo it only gets cooler :p
Lingpeng Kong retweeted
🌟 Thrilled to share our paper, "TreeSynth," has been accepted for a Spotlight presentation at #NeurIPS2025!
🤔 Struggling with repetition & space collapse in data synthesis? Our work introduces 🌳TreeSynth, a novel framework using tree-guided partitioning to generate large-scale, diverse datasets from scratch.
🏆 Models trained on TreeSynth data consistently outperform those trained on human-crafted datasets and other synthetic methods.
See you all at NeurIPS!
🔗 Paper: arxiv.org/abs/2503.17195
💻 Code: github.com/cpa2001/TreeSynth
Lingpeng Kong retweeted
Jinjie Ni @NiJinjie from NUS will be giving a talk titled "Diffusion Language Models are Super Data Learners" at Friday Aug 22 11am HKT. link to talk: hku.zoom.us/j/94293996114?pw…
Lingpeng Kong retweeted
🚀 MiMo‑VL 2508 is live! Same size, much smarter.
We’ve upgraded performance, thinking control, and overall user experience.
📈 Benchmark gains across image + video: MMMU 70.6, VideoMME 70.8.
Consistent improvements across the board.
🤖 Thinking Control: toggle reasoning with `no_think`.
On (default): full reasoning visible;
Off: direct answers, no reasoning ⚡⚡;
❤️ Real‑world user experience: our VLM Arena rating improved from 1093.9 → 1131.2 (+37.3).
More capable, flexible, and reliable in everyday tasks.
Feedback welcome.
🤗 RL Version: huggingface.co/XiaomiMiMo/Mi…
🤗 SFT Version: huggingface.co/XiaomiMiMo/Mi…
#XiaomiMiMo
Lingpeng Kong retweeted
Xinyu Yang from CMU will be giving a talk titled "Multiverse: Your Language Models Secretly
Decide How to Parallelize and Merge Generation" at Friday July 25 11am HKT (Thursday July 24 8pm PDT). Link to talk: hku.zoom.us/j/92651812689?pw…
Lingpeng Kong retweeted
📢 Update: Announcing Dream's next-phase development.
- Dream-Coder 7B: A fully open diffusion LLM for code delivering strong performance, trained exclusively on public data.
- DreamOn: targeting the variable-length generation problem in dLLM!
Lingpeng Kong retweeted
🚀 Thrilled to announce Dream-Coder 7B — the most powerful open diffusion code LLM to date.
