
Building agentic and tiled stuff @UofIllinois | prev @togethercompute @ZJU_China
San Francisco, CA
Joined November 2023
- Tweets50
- Following989
- Followers67
- Likes149
Weili Xu retweeted
While exploring diffusion language models (DLMs), we kept asking ourselves: given speculative decoding, a lossless acceleration technique, why do we still need DLMs? 🤔 Our latest work, ORL, tries to answer this question: AR optimizes intelligence 👆, speculative decoding optimizes speed 👉, and DLMs optimize the Pareto frontier of speed and intelligence 🚀. Through co-design across the algorithms, the training and inference engine, and the kernels, ORL pushes out a new Pareto frontier of intelligence and speed. Similar performance, much faster than speculative decoding. You say lossy spec? ORL-DLM is better too! 😎
🌐 Project + live demos: introspective-diffusion.gith…
Weili Xu retweeted
While QAT helps improve model quality at 2-4 bits, it has a structural optimization problem. Indeed, at every step, the high-precision latent weights receive the gradient updates, but the loss and gradients are computed on their low-bit reconstructions, i.e. the weights the model will use at inference.
This mismatch results in suboptimal training dynamics and a higher loss floor. The main existing QAT methods (learning the scales alongside model parameters, soft-to-hard rounding, or better initialization) do not directly address this mismatch.
With QUASAR, we introduce a simple and computationally cheap way to address the mismatch. QUASAR consistently achieves lower loss than all QAT baselines and adds no inference overhead. Also, it can be combined with most other QAT methods, since it only changes the weight reconstruction map in the fwd pass.
Weili Xu retweeted
Real-time video assistants have to decide when to respond and when to stay silent. Offline scores do not capture this. InteractionBench evaluates this decision for the complete system of model, memory, and response controller, with 1,060 interactions over 812 videos.
We’re teaching a new course at Penn: CIS 6280 · World Models 🌍
cis.upenn.edu/~cis6280/
Every day, LLMs amaze me with one more thing they can solve. But I’ve always felt there’s more to learn than what humans have written down—from observing the world, interacting with it, and experiencing what happens next. To me, that’s what world models are about—and why I see them as one of AI’s next big bets.
However, I struggled to find a systematic course on them—so we are building one!
Topics will include: representation learning, generative models, simulation, model-based RL, video and 3D generation, world models for robotics, reasoning, and code-based world models.
Hands-on work will include:
- Building an environment
- Training a world model
- Learning a policy
and a final research project with leaderboard.
We’re already six lectures in. Slides, demos, and readings are public, and we’ll keep adding materials throughout the fall.
Big thanks to our TA team— @TongMutianTMT @hagsaeng_bag @EnxinSong @KeelyAi04 —for helping bring this course to life!
Weili Xu retweeted
🎉Qwen3.8-27B 迎来最强 4-bit 量化版!
社区发布了基于量化感知训练(QAT)的:QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4
🔥核心亮点
1. 极致压缩
全网络 496 个线性层全面采用 NVFP4(W4A4)量化,模型体积从 55.6GB → 19.7GB,大幅降低显存占用
2. 近乎无损性能
通过蒸馏学习,保留原版几乎 100% 的推理与数学能力,表现远超传统 PTQ 量化方案
3. 即插即用
原生支持 vLLM / SGLang,无需额外转换,直接部署运行
4-bit 不再只是能跑就行,而是开始追求接近原版的性能了!
🎉Qwen 3.8 27B 变身 20GB!Blackwell 架构完美适配版!
社区又又又推出了 Huihui-Qwen3.8-27B-abliterated-NVFP4 最新量化版本!
🔥核心亮点:
1. 体积暴减: 顺应 8 月新发布的 Qwen 3.8 27B 基座,将 55.6 GB 降至 20.0 GB,双 16GB 显卡即可留足 KV 缓存轻松跑起来,无压力
2. 修复长思考 Bug: 完美解决上一版在 Qwen 3.8 标志性的 xhigh 深度思考模式下,因思维奔逸陷入万字复读(thinking-runaway)的致命缺陷
3. MTP 投机采样: 完好保留 bf16 格式的 MTP 团队草稿头,在 RTX PRO 2000 Blackwell 上单流 Prefill 飙到 3,250 tok/s
划重点 部署提示: 基于 vanilla vLLM v0.22.0,直接 vllm serve 自动识别 compressed-tensors,无需额外量化 flag
无论你是想本地调教还是用作生产力
这个版本都是性价比之王
Qwen 牛牪犇!
传送门👇🏻
Weili Xu retweeted
Recently I was reading Jianlin Su’s blog and one statement caught me off guard:
MLA + MTP may actually be a bad combination.
That sounded counterintuitive, so I tried to understand why from the perspective of arithmetic intensity. Following the path from MHA → GQA → MQA → MLA, the result turned out to be surprisingly elegant.
Check Full Blog Here!changyi.fun/posts/attention-…
A short thread on both:
🤖 Made with AI
Weili Xu retweeted
Blog: together.ai/blog/thunderagen…
GitHub: github.com/ThunderAgent-org/…
Paper: arxiv.org/abs/2602.13692
It’s been fun to see how research interacts organically with production this summer at @togethercompute – at San Francisco, CA
Weili Xu retweeted
Cognition used Dynamo to accelerate the rollout phase of agentic RL.
Our work ThunderAgent has also been merged into Dynamo and SkyRL. And achieve lossless speed up up to 4x.
However, current schedulers and KV-cache managers are not yet designed for hour-scale agentic rollouts, leaving significant room for improvement in trajectory-aware scheduling, KV lifecycle management, and distributed memory pooling.
paper: arxiv.org/pdf/2602.13692
code: github.com/ThunderAgent-org/…
web: thunderagent.ai/
Weili Xu retweeted
Our research team has 9 papers at ICML next week!
Spanning the full stack from frontier agents to GPU kernels, we're excited to share what our researchers and collaborators have been working on.
If you're in Seoul for ICML come meet our team or catch a talk! 🧵
Weili Xu retweeted
2/ ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System
Paper: arxiv.org/abs/2602.13692
Weili Xu retweeted
Multi-GPU kernels are the real test for coding models. Today at @aiDotEngineer, @simran_s_arora shared ParallelKernelBench, an open-source benchmark for evaluating whether LLMs can write fast CUDA kernels for real communication-heavy workloads.
Proud to see this work from the Together AI Frontier Performance team.
Weili Xu retweeted
A message to Anthropic leadership: You're not special. Making sure AI goes well is a team effort not a "you effort."
Weili Xu retweeted
Labs starting to pull up the ladders on the ability to diffuse AI was inevitable. Doing it without telling the user is misaligned.
Weili Xu retweeted
Very sad news for the LLM research and open-source community. Does this mean PhD researchers in frontier LLMs, or contributors to open-source LLM infrastructure like Megatron, FSDP, Verl, SGLang, and vLLM, may be using a degraded Claude model in their daily work without being notified?
As part of Dynamo 2.0, the program abstraction proposed in ThunderAgent is being standardized as part of the nvext.agent_context protocol in dynamo.
Inference scheduling/KV cache management with agent lifecycle awareness isn't future anymore, it’s the trend happening right now!
Excited to share that ThunderAgent has been integrated into NVIDIA Dynamo as an experimental router for agentic workloads!
ThunderAgent was designed to schedule at the granularity of agent runs, making agentic serving/rl upto 4x faster!
Huge thanks to @0xishand , @KranenKyle , and the Dynamo team. They have been exceptionally efficient and proactive — the team had already started pushing this forward even before I officially joined @nvidia .
Looking forward to seeing ThunderAgent ideas further evolve within Dynamo. And thanks for the help from @togethercompute
Link: github.com/ai-dynamo/dynamo/…
@simran_s_arora @Chenfeng_X @_weilix @yinfang_chen
#AI #MLsys #Agent #Nvidia
fun part in the post: #OpenSourceAl😅
📢 Official Announcement: Qwen Partners with Fireworks AI to Accelerate Access to Qwen Family Models
We are pleased to announce a strategic partnership between Qwen and Fireworks AI to deliver optimized, production-ready deployment of Qwen's closed weights models via the Fireworks Platform. @FireworksAI_HQ
This collaboration empowers developers and enterprises to:
✅ Deploy Qwen models with lower latency and reduced fine tuning and inference costs
✅ Leverage enterprise-grade reliability, security, and scalability
✅ Integrate seamlessly into modern AI workflows
🔹 Get started with Qwen on Fireworks: app.fireworks.ai/models/fire…
#Qwen #FireworksAI #OpenSourceAI #LLM #AIInfrastructure #ResponsibleAI #DeveloperCommunity
Weili Xu retweeted
Big Update🤩: #paperclip now includes full papers from all of arXiv, PubMed Central and 150 million abstracts!🖇️
You can give your LLM all that knowledge in one line—all optimally indexed for AI agents. Much more thorough and ~100x faster than web search, and free.
Weili Xu retweeted
Introducing Kimi K2.6 from @Kimi_Moonshot, a multimodal agentic model with Agent Swarm scaling to 300 sub-agents and long-horizon coding stability. AI natives can now use Kimi K2.6 on Together AI and benefit from reliable inference for production-scale autonomous agent workflows.

































