Research Scientist @AdobeResearch | Ph.D. @ CUHK | Prev. Research Intern @AdobeResearch | Photographer | Opinions are my own
California, USA
Joined November 2019
- Tweets255
- Following520
- Followers774
- Likes2.1K
Pinned Tweet
Excited to share Chimera!
As Kimi K3 brings hybrid linear attention into the spotlight, our work explores the same architectural transition from the visual side: adapting KDA-based hybrid attention to token-extensive visual generation, together with a systematic scaling recipe.
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch
Tianyu (Steve) Wang retweeted
🪄 Introducing OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing
🧙 Any number of references, any modality, any attribute, any combination. OmniXRef transfers each into one coherent edit. (1/6)
Tianyu (Steve) Wang retweeted
HERE IT IS. The Next Generation Roadster developed by SpaceX and Tesla!
There is a fan-suction system at the bottom of the car that can pull up to 2 gs allowing it to stick upside down.
It just flipped UPSIDE DOWN with cold gas thrusters and landed. HOLY SHIT.
Tianyu (Steve) Wang retweeted
新时代做research太累了， 任何解决”简单“问题的paper都会在两个月内过时。祝愿大家都能找到重要问题 把时间花在重要的问题上
What is the role of academic computer vision research in the age of increasingly powerful large models? Is GPT-6 Astra a step change? How can a researcher have an impact today in academia?
These are the questions I ask myself as I head off to ECCV 2026, a conference I’ve attended since 1992. One of my papers this year is VIGA, a method that takes an image as input and outputs a 3D Blender scene that represents that image. This is a classical inverse-graphics task and VIGA was the first method to solve it using an agentic approach.
The idea is now several years old and the first version of the paper was rejected. This delayed publication significantly. After it was accepted at ECCV, it was quickly surpassed by people using Claude Code for the same purpose. Today GPT-6 Astra blows away all previous results. But we still head off to ECCV to tell the community about our invention that is now fully out of date.
The way academic work often progresses is that one reads recent papers, notices that they have limitations, comes up with a new idea, explores this, publishes it, etc. Any published paper I read today is based on ideas that are at least a year old. And those ideas were based on the literature of the time, which was also a year old. That means that any paper I see at ECCV is likely two years out of date. In AI today, two years means your work is likely irrelevant.
At CVPR this summer I noticed that many authors have not gotten the message. They continue to work on “old” problems that have a long history. This history is based on assumptions about how the “vision problem” will be “solved”. The truth is that it is being solved in a very different way and many of these problems are no longer relevant. Another group of papers focuses on very niche problems where large models likely fail because of insufficient data or lack of business interest. The impactful papers were largely from industry and had long author lists and massive data+compute behind them. These papers were also out of data, describing systems that had been released months before, but at least they served to provide the community with more complete documentation and analysis of commercial systems.
So what should academics do? First, we need to put aside the tools we’ve used for years and start from scratch. Every project should start by trying really hard to solve the problem with existing tools. I would like to see every paper begin with a detailed experimental analysis of how existing models perform and why they fail (if they do). This gives the kind of insight we need today. Then, assuming current models fail, the solution should provide some fundamental insight that will outlive the next release of such models.
Reviewers today still focus on technical novelty. This pushes people to focus on tweaking architectures rather than clearly moving the field forward. Papers need to be judged based on their novel insight and not their novel technical contribution. This is a real shift in thinking but it focuses us on what matters - progress of the field.
If we want there to be a “field” of computer vision, then it can’t become a marginal backwater, focusing on esoteric problems. If you haven’t tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework. This omission should be seen as negatively as not having a previous work section.
Concretely, I think papers should include a new section analogous to “Related Work” where that related work is current models and how they perform on the task. Reviewers should start asking for this and expecting authors to be able to articulate their insights about the limitations of existing large models.
I'm interested in your thoughts.
Tianyu (Steve) Wang retweeted
I recently completed a research project through a highly automated collaboration with Codex. The division of labor was simple: Codex maximized exploration, while I focused on judgment.
In detail, Codex proposed hypotheses, implemented and debugged methods, ran experiments, analyzed results, and iterated from feedback. I set the research direction, chose the questions, inspected the evidence, and decided which ideas were worth pursuing.
This is also a concrete example of the research workflow I discussed in my previous post. I believe it is particularly effective when a problem is close to purely logical and each hypothesis can be verified with modest compute. Agents can then explore at high throughput, while cheap verification enables rapid iteration.
The result is my new paper, which asks a question: what limits length extrapolation in recurrent models, and how can we learn recurrent dynamics that remain effective beyond the training horizon?
More specifically, recurrent models are coming back not only because they are efficient. Their shared update rule provides a structurally natural path to length extrapolation. Yet learning recurrent dynamics that remain reliable over substantially longer horizons is still an open problem.
To study this problem, I build a controlled task suite that separates state tracking, memory retrieval, and their interaction. My diagnostics show that vanishing parameter gradients (previous focus) alone do not explain extrapolation failure. A fixed transition rule can still be learned from dense local supervision despite severe gradient decay. The harder failure occurs when future losses cannot transmit sufficient credit to earlier recurrent states that must preserve useful information.
This suggests intervening before the parameter gradient is formed. I call the gradient of future losses with respect to an earlier recurrent state as state credit. It is the upstream learning signal: a future loss must first reach the state before it can train the recurrent update.
This motivates Credit Stabilization through Time (CST) to act directly on this signal. During backward propagation, it restores the norm of state credit when contraction accumulates across sequence chunks, while preserving its direction and leaving the forward computation unchanged.
The results support this hypothesis across two regimes:
Synthetic tasks
• CST significantly improves length extrapolation across all task families and evaluated settings, with gains verified at up to 128× the training length.
Real language modeling (still toy)
• I train a 123M-parameter recurrent language model at 4K and evaluate it up to 128K on both in-domain and out-of-domain data.
• CST produces a modest improvement over all tokens, but a substantially larger gain on key tokens that require long-range context.
• The benefit generally increases with extrapolation length.
Learning Length-Extrapolatable Recurrent Models
Paper: arxiv.org/abs/2609.09157
This is my first paper centered on model architecture design and language modeling, and also my first attempt at this kind of highly automated research workflow. So comments and corrections are genuinely welcome. Hope you find both the paper and workflow make sense :)
Tianyu (Steve) Wang retweeted
Thanks so much for sharing! @_akhaliq 🥰
The core idea is to learn how the world evolves, rather than directly predict what the future looks like. LDR maps past frames into structured latent states and rolls them forward with kinematic integration, learning only the higher-order motion residuals.
To our knowledge, this is the first video world model that learns the underlying dynamics purely from pixels and extrapolates them beyond the training distribution.
- Paper: huggingface.co/papers/2608.0…
- Code: github.com/adobe-research/LD…
- Model: huggingface.co/haodongli/LDR
- Data: huggingface.co/datasets/haod…
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Thanks so much for sharing! @_akhaliq 🥰
Paper / code / models / data are now public, check them out! 🚀
- Paper: huggingface.co/papers/2608.0…
- Code: github.com/adobe-research/LD…
- Model: huggingface.co/haodongli/LDR
- Data: huggingface.co/datasets/haod…
Learning How the World Evolves
Extrapolative Video World Models via Latent Dynamics Reasoning
paper: huggingface.co/papers/2608.0…
Tianyu (Steve) Wang retweeted
Excited to share Chimera! My favorite part is mShortConv: a simple way to tell the model that text is 1D, images are 2D, and videos are 3D—even after they are flattened into the same sequence. Together with KDA, this enables a single scan and NoPE. Proud of the team!
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch
Paper: arxiv.org/abs/2607.28611
Tianyu (Steve) Wang retweeted
Interested in how frontier labs pre-train image/video generation models?
We were too.
Since those recipes are rarely made public in full, we started from the most mature pretraining playbook available in the open: how modern LLMs are built.
Introducing Chimera: a visual generation model family that brings LLM-style hybrid linear attention and scaling co-design to visual generation.
In large-scale pretraining, nearly every design choice eventually shows up.
That means solving architecture design and scaling as one coupled problem: every architectural choice changes how the model scales, and scaling behavior determines which choices actually survive.
Chimera approaches both jointly, building a model family that remains predictable as model size, compute budget, and data distribution change.
Our key architectural observation is a simple division of labor: a single raster-ordered KDA stream carries long-range state, while modality-aware short convolutions preserve native local geometry.
Together, they form an effective and elegant linear-attention backbone for multidimensional visual data, with periodic MLA providing direct global interaction and sparse MoE expanding capacity at controlled activated compute.
This design comes with a useful effect: NoPE.
In Chimera, position is represented by the computation itself. KDA’s ordered recurrence and learned state decay encode order and recency, while the short convolutions encode local spatial-temporal structure.
Explicit positional embeddings are not needed in our design.
Because these mechanisms are not tied to a fixed training grid or sequence length, Chimera shows strong zero-shot extrapolation in both space and time.
Trained exclusively on 1K images and 5-second videos, it directly generates coherent 4K images with little visible quality degradation and 30-second videos with only 6.5% FID degradation over the final five seconds, all zero-shot, without resolution- or length-specific finetuning.
But architecture alone is not a pretraining recipe unless it scales predictably.
Thus, scaling should not be treated as an afterthought: fitting a curve over model sizes is easy; making that curve meaningful is much harder.
If every model size is differently under-tuned, your scaling law may simply be measuring optimization error.
We propose HeteroP to transfer proxy-tuned hyperparameters module by module across width and depth, giving us a consistently tuned model family. This allows us to fit Chinchilla-style laws over activated model size, training tokens, and the image-video data mixture.
The laws not only provide the reference for compute-optimal model and data size, but also suggest that visual generation may be more model-hungry than we tend to assume.
Under the same parametric loss-fitting method used in Chinchilla, compute-optimal model size grows as FLOPs^0.516 for images, compared with FLOPs^0.46 for language. Video is even more model-hungry, with the exponent rising from 0.516 for images to 0.544 for video.
Guided by these laws, we trained an 11B-parameter Chimera that activates only 2B parameters per token, using ~600 H100 days.
- It matches Wan-2.1 2B pretraining loss with 7.3x fewer FLOPs, and runs 2.14x faster than full attention at 255K tokens.
- It matches FLUX.1-dev and Z-Image-Turbo on GenEval and outperforms both on DPG-Bench, using roughly 20× less training compute than Z-Image-Turbo.
Ultimately, Chimera indicates that once you pretrain at scale, every decision shows up.
And if the Kimi K3 recipe caught your attention, Chimera may look oddly familiar, except the tokens are pixels and frames with diffusion models.
A team effort from team @ChongjianG30781 , me, @VisionSteve , Jiuxiang Gu, @Xu_Arthas , @chenziwee , @ShaotengLiu , @Jingorz , @YicongHong , @Zefan_Cai , @HaoTan5 ; supported by Hailin Jin and @kalyank_s at @Adobe @AdobeResearch
Great work! But it would be great to cite our 2D-Box ObjectMover paper in CVPR 2025.
Here is a short video of our Thinking in Boxes. Enjoy!
More results on our website: thinking-in-boxes.github.io/
Tianyu (Steve) Wang retweeted
1/ Our new @reve image model is now #2 on the @arena text-to-image leaderboard — behind only GPT Image 2, ahead of Nano Banana Pro, Microsoft, xAI and everyone else.
And it's a 125 point jump over Reve 1.5 from just 3 months ago.
The research story behind it 🧵👇
Tianyu (Steve) Wang retweeted
🚀 Excited to release LongLive 2.0!
🎬 An end-to-end infrastructure for long video generation, with FP4 and parallelism at the core of both training and inference.
⚡45.7 FPS generation speed on 5B model⚡
✨ LongLive 2.0 supports real-video training, few-step distillation, multi-shot training/inference, sequence-parallel acceleration, NVFP4 KV cache, and async VAE decoding deployment.
🧩 To our knowledge, this is the first open-source 4-bit long video generation infra that covers both training and inference.
🙌 Welcome to check it out, try it, and share feedback!
🔗 Code: github.com/NVlabs/LongLive
📰 Paper: huggingface.co/papers/2605.1…
🎥 Demo: nvlabs.github.io/LongLive/Lo…
#LongVideoGeneration #VideoGeneration #Realtime #AIInfra #EfficientAI #FP4 #Parallel #NVIDIA
RT @BoyuanChen0: If there is one secret recipe about GPT Image 2, it must be codex. In the last 6 months, codex has been zero-shoting every…
If you’re attending, please stop by our oral session for EditVerse:
I won’t be there in person, but I’d be very happy if you could check out the work, chat with the team, and join the discussion.
📍 Room 201 A/B
🕚 Apr 25, 11:18–11:28 AM
Excited to share our paper EditedVerse is accepted as oral to ICLR 2026! Many thanks to our amazing coauthors!!
Paper Link: arxiv.org/pdf/2509.20360
Project Page: editverse.s3-website-us-east…
Tianyu (Steve) Wang retweeted
Excited to share our paper EditedVerse is accepted as oral to ICLR 2026! Many thanks to our amazing coauthors!!
Paper Link: arxiv.org/pdf/2509.20360
Project Page: editverse.s3-website-us-east…