PhD Candidate @HKUSTGuangzhou｜@NVIDIA Research Intern
Guangzhou | HK, SAR
Joined October 2022
- Tweets1.2K
- Following1.5K
- Followers504
- Likes1.4K
Owen Tian Ye retweeted
This "neural render" approach is the future of gaming and VR
🚀 Excited to introduce StreamRender-H3 — bringing real-time interactive worlds to life with diffusion models!
Built on Sol-Engine, we post-trained @MiniMax_AI H3 Ref2VA into a real-time streaming neural renderer.
🎮 Code drives the world. A game engine handles physics, object states, and player interactions, while H3 generates the visuals.
⚡ Real-time streaming: 2 denoising steps, 24 FPS, and ~273 ms per chunk on 8×GB200 GPUs.
🎨 One world, endless appearances: Change the reference image and prompt to render the same interactive world in entirely different styles.
🔥 Powered by streaming post-training, distillation, a fast causal H3 VAE, and full-stack inference acceleration.
Check out our demo and code! 👇
demo: nvlabs.github.io/Sana/Sol-En…
code: github.com/NVlabs/Sana/tree/…
🚀 StreamRender-H3：Interactive Coding Worlds with Real-time Streaming Diffusion Render
World = Code + Diffusion Render !
🚀 Excited to introduce StreamRender-H3 — bringing real-time interactive worlds to life with diffusion models!
Built on Sol-Engine, we post-trained @MiniMax_AI H3 Ref2VA into a real-time streaming neural renderer.
🎮 Code drives the world. A game engine handles physics, object states, and player interactions, while H3 generates the visuals.
⚡ Real-time streaming: 2 denoising steps, 24 FPS, and ~273 ms per chunk on 8×GB200 GPUs.
🎨 One world, endless appearances: Change the reference image and prompt to render the same interactive world in entirely different styles.
🔥 Powered by streaming post-training, distillation, a fast causal H3 VAE, and full-stack inference acceleration.
Check out our demo and code! 👇
demo: nvlabs.github.io/Sana/Sol-En…
code: github.com/NVlabs/Sana/tree/…
Owen Tian Ye retweeted
🚀 SoL-Refiner: 2K video in one step.
Give it a low-resolution video from any generator, and SoL-Refiner turns it into sharp, detailed 2K/4K video with a single denoising step.
🤗 Hugging Face: huggingface.co/papers/2609.3…
🎬 Demos & Page: nvlabs.github.io/Sana/Sol-Re…
Owen Tian Ye retweeted
"DiffusionGemma as Jev" showcases the power of non-autoregressive architectures.
While Jev demonstrates the value of rapid decision models, running DiffusionGemma in this paradigm leverages canvas diffusion to evaluate structured choices in a single parallel pass:
⚡ ️Massive Parallelism: Denoises across an open canvas in a single step instead of sequential autoregressive token generation (~0.2s on a DGX spark).
🧠 Full Bidirectional Attention: Allows every option to attend to the full context concurrently, yielding well-calibrated decision distributions.
👁️ Multimodal Grounding: Inherits Gemma 4's spatial vision capabilities for complex visual and text decisions.
Read more about this approach here:
github.com/vllm-project/vllm…
xitter.cf/mmastrac/status/210037…
xitter.cf/mmastrac/status/210062…
Owen Tian Ye retweeted
thoughts on Typesafe/Jev: I'm surprised that a general-purpose classifier (or discriminative model) can be just as interesting to the public as a general-purpose generative model. I've worked on search and discriminative models for years, so maybe I should have called this upfront, tbf I never thought about making them general-purpose beyond search. Typesafe also positions Jev really well as "System 1", a complement to the generative models we already have. Humble, unassuming, and sidestepping the frontier-lab warzone.
Jev's success could be disruptive to today's agentic systems in many ways, such as tool-calling and routing patterns. Over the last three years the consensus was to use a small generative model for routing, tool calling, MCP etc. Time to pause and rethink such architecture, because we'll probably see tool calling and routing move back to discriminative models. Whoever ships the next Jev-like open-weight base model will likely win the community.
🚀 We’ve released the full SoL-Pi report!
It includes the complete methodology, extensive experiments, scaling studies, and detailed analysis behind SoL-Pi.
Would love your feedback — and an upvote if you find it useful! 🙌
huggingface.co/papers/2609.2…
Owen Tian Ye retweeted
Real-time AI video is going spatial.
With Vidu S2, we demonstrate—to our knowledge—the first system for real-time interactive spatial video generation and editing for VR.
Imagine speaking to an AI character and feeling like they’re right there with you.
The character responds as the video generates, with synchronized left- and right-eye views that create depth and presence.
Transform a live video stream: change its style, clothing, characters, or surroundings, and experience the result in stereo.
A new kind of entertainment: immersive visual experiences shaped by your interaction.
What would you build with this? Explore Vidu S2-Avatar and Vidu S2-Editing below. Spatial video access is coming in a future release.
Try it: vidu.com/vidu-stream
API: platform.vidu.com/vidu-strea…
Technical report: arxiv.org/pdf/2609.11638
Owen Tian Ye retweeted
🚀 Sol-H3 on DGX Spark: 768p in Under a Minute 🤩
Monday: 8×B300, 5s 768p in 1.65s — faster than playback.
Today: the same stack on one desktop Spark — about 56s hot E2E.
Five seconds of 1344×768 video at 24 FPS with stereo audio, on a single NVIDIA DGX Spark (GB10).
Two-stage, not the datacenter profile:
384p H3 draft → latent ×2 → H3-to-LTX VAE adapter → 768p LTX refine → VAE decode
No decode/re-encode between stages. Stage 2 is conditioned on the draft latent, so Gemma stays off the box. Quantized weights stay resident; sparse attention cuts the rest.
Stage 1 takes any MiniMax-H3 few-step LoRA. Timing is hot E2E (encode → both stages → video/audio VAE); cold start and MP4 mux are separate. Apache 2.0.
Server was realtime. Edge is one box, under a minute.
🔗 nvlabs.github.io/Sana/Sol-En…
Amazing team effort—full credits in the blog.
@haopengl33 @lawrence_cjs @yitongli165665 @shanasaimoe ,Jingyu Xin, @HaochengXiUCB @songhan_mit
Owen Tian Ye retweeted
🚀 Introducing SoL-Pi!
⚡️ RSI for Efficiency. Efficiency for Swarm Intelligence.
❓ Before scaling to thousands of parallel agents, can we make each agent waste fewer tokens?
🔧 SoL-Pi improves the harness layer, the software coordinating models, tools, context, and multi-agent collaboration. Agents build experimental environments from public repositories and verifiable tasks, then iterate:
Inspect trajectories → Propose ideas → Modify the harness → Experiment → Verify.
📊 Across 535 executable environments, agents explored 152 initial directions, extending promising ideas through independent auto-research loops. Only four mechanisms survived the filtering process.
🏆 The results:
▪ EdgeBench: 45%–49% fewer tokens and roughly one-third lower cost than Pi.
▪ Terminal-Bench 4, 63 CPU-only tasks: 26.3% lower total model cost and 11.6% lower cost per solved task than Pi, with reduced task success.
▪ Agent Swarm: one main agent coordinating 20 parallel SoL-Pi agents on kernel optimization used 26.8% less model API spend than the standard Pi swarm.
🔮 Next, we’ll use SoL-Pi to run larger, cheaper research loops and discover further efficiency improvements: Efficiency for Efficiency, with compounding returns. Humans set the initial direction; agents explore and verify autonomously; humans then interpret the findings and refine the code.
✨ Intelligence for Efficiency. Efficiency for Intelligence.
🌐 Official Blog
Read the blog → nvlabs.github.io/SoL-Pi/
💻 Official Code
Code → github.com/NVlabs/SoL-Pi
Owen Tian Ye retweeted
🚀 Introducing SoL-Pi!
⚡️ RSI for Efficiency. Efficiency for RSI
🔮 We observe scaling laws in RSI loops.
🌐 Official Blog
Read the blog → nvlabs.github.io/SoL-Pi/
💻 Official Code
Code → github.com/NVlabs/SoL-Pi
🚀 Introducing SoL-Pi from NVIDIA.
⚡️ RSI for Efficiency. Efficiency for Swarm Intelligence.
Before scaling to thousands of parallel agents, can we first make each agent waste fewer tokens?
SoL-Pi targets the harness layer for Agent !
1/8
Amazing work by our team — @HaoZhe65347, @QihangCao11, and @gao_sensen40956 — with huge credit to our team leader @xieenze_jr for leading the team and driving this work forward! 🚀
🔮 What’s next?
Use SoL-Pi to run larger + cheaper research loops—and let those loops discover further efficiency improvements.
Efficiency for Efficiency, with compounding returns.
Humans set the direction → agents explore + verify → humans interpret + refine the code.
7/8
✨ Intelligence for Efficiency. Efficiency for Intelligence.
Harness-level RSI can be a practical path toward scalable agent research: improve each agent first, then scale the swarm.
🌐 Blog: nvlabs.github.io/SoL-Pi/
💻 Code: github.com/NVlabs/SoL-Pi
