AI Application Architecture
Dolomiti
Joined November 2009
- Tweets734
- Following171
- Followers14
- Likes504
慎为 retweeted
vLLM 开始支持双向 KV 传输，针对多轮 Agent 在 prefill/decode 分离部署中反复计算历史上下文的问题：上一轮 decode 节点算出的 KV，下一轮可以传回 prefill 节点继续复用。
在 AWS P5en + EFA、一个 prefill 节点配一个 decode 节点的测试中，GLM-5-FP8 在约 20K 输入长度下，首 token 延迟从 2,334 ms 降到 702 ms。但这个对照关闭了 prefill 端的前缀缓存，用来模拟缓存失效或请求换节点。
短上下文里，搬运和协调的开销可能比重算更大。
官方介绍：vllm.ai/blog/2026-10-12-bidi…
慎为 retweeted
Sales de la burbuja tech y te das cuenta de que nadie sabe qué coño son los agentes de IA, MCP, Opus, Astra, skills… viven todos tan tranquilos 💆🏻♀️💆🏻♀️
慎为 retweeted
⚡ Towards the Next FlashAttention
🧪 Squeezing out performance from GPUs now requires programming at the assembly level (like in FlashAttention/MLA/KDA etc). Can AI agents do it?
📏 That’s the question behind 𝗣𝗧𝗫𝗕𝗲𝗻𝗰𝗵. On H100 and B200, we ask models to write matrix multiplication and attention kernels using architecture-specific PTX: assembly-like instructions that expose each GPU’s specialized hardware (tensor cores, TMA, TMEM, ...).
🔍 PTXBench check three things:
• Is the answer right?
• Is the kernel fast?
• Did the requested instructions actually run?
🔴 The gap exists. An LLM can use the right instructions but still produce a slow kernel. Attention backward is especially hard, and no model we tested consistently matches expert libraries across the suite.
🛠️ We also built 𝗙𝗶𝘅𝗶𝘁, an SFT recipe that trains on the model’s own failures. Each example pairs a broken kernel and profiler feedback with a teacher repair and synthesized reasoning.
🟢 After Fixit, Qwen3.6-27B solves Hopper GEMM and attention tasks it couldn’t solve before. New shapes and attention variants are still hard.
🧠 Big questions remain:
• Should agents write PTX directly, or will GPU DSLs scale better?
• Correct, fast PTX kernels are hard to collect. What’s the minimal supervision required to train a PTX expert?
• Can a model trained on one GPU architecture generalize to another?
• How do we get from the right instructions ran to this thing is actually fast?
🚀 Getting PTX right is step one. Making it fly is what's next.
📖 Paper & Blog: arxiv.org/pdf/2608.17379 & zhang677.github.io/blog_md/p…
💻 Code: github.com/zhang677/PTXBench
🍾 Weights + data: huggingface.co/collections/G…
Amazing collaboration with @yi_xin_dong Chengze Fan @zz30gs Yueming Yuan @ShaoweiZhu95pu and @KunleOlukotun! Huge thanks to the generous support from @sgl_project @tinkerapi and Gemini Academic Program Award!
慎为 retweeted
🚀 Kernel Design Agents to Optimize Kimi Delta Attention (upto 2.96x)
When the KDA (Kernel Design Agents) project was initiated, its name happened to coincide with another KDA (Kimi Delta Attention). More than once, people asked us , "Can we use KDA to write a version of KDA?" Today, right at a moment with a full moon good for shooting, we are thrilled to share the latest results of KDA(gent)-v0.6: using KDA(gent) to optimize KDA(ttn)!
KDA-v0.6 achieves this through
* Humanize2 Flow & LLM Collaboration Optimization: Introducing multi-round iterative evolution with Flame Chase, GPT-5.6-Sol, and Fable-5, paired with an automated Workspace cleanup mechanism to significantly broaden the operator search space.
* Multilingual Domain-Specific Skill Stack Support: Fully supporting CuTe-DSL, CUDA C++, as well as recent Agent-Native CAKE IR and TIRx. Combined with customized diagnostic tools (such as CuTe's dedicated IKET pipeline analyzer and TIRx's CPU-side numerical simulation and static checks), achieving deep performance bottleneck diagnostics.
* Self-Evolving Kernel Wiki: Building a dynamically updated knowledge base that automatically cleans up invalid/erroneous implementations, corrects classification tags, and streamlines search indexing, significantly improving the Agent's knowledge retrieval quality and code generation accuracy.
KDAgent ultimately optimized the KDAttn kernel to 2.96x of FlashKDA, and we face a lot of unexpected hacks during the optimization. More details shared in our blogs nvlabs.github.io/kda/blog/20…
Release results of KDA for KDA: github.com/NVlabs/kda/tree/2…
Community & Production Deployment:
All kernels are open-sourced for community verification! For deploying NVIDIA Agentic CUDA acceleration in production environments, we recommend using the production-level version rigorously verified by FlashInfer.
By the Team: Dongyun Zou, Yixin Dong, Hongyi Jin, Junxian Guo, Yahui Cui, Avery Huang, Zihao Ye, Junru Shao, Changye Li, Zijian Zhang, Sihao Liu, Song Bian, Ligeng Zhu.
慎为 retweeted
1/5 New rLLM Blog Post: Continual learning for agents 🧵
Can an AI agent keep learning after deployment—from real production traffic?
Each interaction yields one trajectory. Unlike build-time RL, you typically can't sample multiple rollouts for the same prompt, so GRPO's group-relative advantage no longer applies.
Our result shows a +16.2 points improvement on real SWE tasks.
Full post: rllm-project.com/post.html?p…
慎为 retweeted
GPT-6 Astra attempted harmful actions 97% of the time when it was asked to stab a human-like figure, heat compressed gas, or produce toxic fumes, succeeding in 62% of its attempts. Fable 5.1 refused more often, attempting 80% of trials and completing 34%.
慎为 retweeted
用 AI 就能把色卡换装视频做得一模一样，真没那么难。
使用Hypit，在codex 里直接就能一比一复刻出来。
天凉了，正好让模特换上秋装。创意来自 @原来是陶阿狗君
🤖 Made with AI
卧槽，Hypit 这次确实有点意思。
把它接进 Codex，丢一条参考视频过去，再说一句“帮我 Clone”，它就能把原片拆成可编辑的视频工程。
人物能换、服装能换、色卡能换。
原片的构图、镜头节奏、字幕、转场和画面层级，会尽量保留下来。
我这次用 Codex 跑了 Hypit，复刻了两条短视频。
一个是「豆包帮我清理桌面」的场景创意 @Yanhua，一个是 @陶阿狗君 的旅行变装。
它先把视频拆成一个可编辑的 workflow。
谁在说话、哪一段用什么图、字幕怎么排、画面怎么动，都变成工程里的零件。
流程很简单：
1、给 Codex 装上 Hypit Skill
npx skills add hypit-ai/hypit -g
2、配好视频模型 API
我这里用即梦 CLI 调 Seedance，先确认模型能正常出片。
3、把参考视频和人物图交给 Codex
告诉它保留原片构图和分镜节奏，把人物换成参考图角色，再生成新的色卡和服装。
4、先确认图片，再生成动态镜头
图片、视频和声音可以分阶段处理。图片不满意先改，确认后再调视频模型出片。
我觉得 Hypit 最有意思的地方，是它把视频制作从一次性「抽卡」，变成可以持续修改的项目。
你能一边预览成片，一边看对应素材和源码。想换人、改字、调镜头节奏，都有地方下手。
工具是开源的，模型和 API 费用各自另算。
想用 AI Agent 做视频 Clone 的，可以自己装了试试，效果因模型和参考片差异会比较大。
官方仓库（开源）：
github.com/hypit-ai/hypit
慎为 retweeted
🚀 Sol-H3 on DGX Spark: 768p in Under a Minute 🤩
Monday: 8×B300, 5s 768p in 1.65s — faster than playback.
Today: the same stack on one desktop Spark — about 56s hot E2E.
Five seconds of 1344×768 video at 24 FPS with stereo audio, on a single NVIDIA DGX Spark (GB10).
Two-stage, not the datacenter profile:
384p H3 draft → latent ×2 → H3-to-LTX VAE adapter → 768p LTX refine → VAE decode
No decode/re-encode between stages. Stage 2 is conditioned on the draft latent, so Gemma stays off the box. Quantized weights stay resident; sparse attention cuts the rest.
Stage 1 takes any MiniMax-H3 few-step LoRA. Timing is hot E2E (encode → both stages → video/audio VAE); cold start and MP4 mux are separate. Apache 2.0.
Server was realtime. Edge is one box, under a minute.
🔗 nvlabs.github.io/Sana/Sol-En…
Amazing team effort—full credits in the blog.
@haopengl33 @lawrence_cjs @yitongli165665 @shanasaimoe ,Jingyu Xin, @HaochengXiUCB @songhan_mit
慎为 retweeted
GPT6 Astra solved Rubik’s Cube with robot hands
dex-rubik-cube.yanjieze.com/
慎为 retweeted
This shot walks through seven rooms without a cut.
GPT-6 Astra mapped the space, the blocking never left the viewport, and Higgsfield Seedance 2.5 rendered the whole run.
慎为 retweeted
wtf！？视频拍完以后，居然还能重新选机位。
早上写过李飞飞 World Labs 新发布的世界模型 Atlas，这个是刚放出来的一个具体玩法。
现场只用 3-5 台普通手机/运动相机，从几个角度拍下砸西瓜的过程。
Atlas 会先理解并重建这个动态场景，然后直接冻结时间，把虚拟摄影机移动到现实里根本没有摄像头的位置，重新渲染出新的镜头。
相当于几台手机，就能做以前需要一整套多机位阵列才能完成的 Bullet Time。
我觉得这个 Demo 特别能说明世界模型和普通视频模型的区别：
普通视频模型是在生成一段视频，世界模型是在重建这段视频背后的世界。
以后可能真的是先把事情拍下来，至于摄影机到底放哪，回去再决定。
太夸张了这个效果🤯
慎为 retweeted
当很多公司都在哀嚎 AI 账单贵到用不起，Uber 刚发的这篇技术长文直接把全行业看傻了:
全公司 70% 的代码 PR 全由 Agent 接管，调用量半年狂飙近 10 倍，
但总 AI 账单被死死按在原地一分没涨，单次会话成本硬生生砍掉了 52%，
他们直接把软件工程做成了一座全自动的软件工厂，真的牛逼，
以前是人打开聊天框求 AI 帮一把，
现在 Uber 养了一整支托管 Agent 舰队: 审代码、修挂掉的 CI、分诊报警全是专职 AI 自动上班，
人变成了抽查质检的厂长，每天跑 3 万多次流水线任务
他们最狠的地方，是把 AI 账单拆成了一道极其冷酷的乘法题，
用量可以疯涨，但零价值的废 Token 必须被物理抹杀:
1️⃣ 规划用大脑，干活全用便宜子 Agent:
任务拆解用顶级大模型，具体搬砖的子任务全部默认切给轻量模型，这一招直接成了全厂杠杆最高的开关；
2️⃣ 把 5 分钟缓存改成 1 小时:
工程师去倒杯水开个会回来，上下文缓存依然在，原价 1 折直接续上，40 万 token 强制写摘要压缩，绝不把整本历史无限复印；
3️⃣ 别把一千种工具说明书塞进大脑:
1000 多个内部 MCP 工具统一走网关，模型要用时先搜索再按需挂载，绝不在开局就傻傻灌入 7 万 token 的工具定义；
4️⃣ Code-mode 砍掉话痨轮询:
跑 SQL 和批处理让模型自己写脚本去跑，只交回最终结果，避免一问一答的无效 ping-pong，单项 token 直接暴省 90%；
5️⃣ 2400 万节点知识图谱导航:
给 Agent 发一张包含全公司服务和事故的活地图，以前翻代码盲搜 20 分钟还报错，现在 38 秒精准定位
用量涨不可怕，浪费涨才可怕，
从人机对话迈向托管工厂舰队，大厂把 AI 真正落地的账本，算是彻底算明白了
