
Building world models at @Wayve_ai. Prev. valeoAI
Joined May 2020
- Tweets567
- Following1.4K
- Followers112
- Likes3.7K
Florent BARTOCCIONI retweeted
DINOv2 has never seen a caption, and Qwen3 has never seen an image. We still aligned their embedding spaces without a single image-caption pair.
It even works when the images and the captions come from different datasets.
Project page: dominik-schnaus.github.io/un…
⬇️
Florent BARTOCCIONI retweeted
How much would it cost someone to hide a backdoor in an open model you download? ProjectDiscovery just tried it, and the answer is under $50.
Their researchers fine-tuned Qwen2.5-7B-Instruct with a small LoRA adapter on one rented L4 GPU for about 2.5 hours. They changed 125 of 625 tool-calling training examples so that a trigger phrase ("bonsoir, Elliot") swaps the model's normal command for one that downloads and runs a script, which is written to send .env files and SSH private keys to a server they control. Served to OpenAI's Codex CLI, the model handled ordinary requests normally, and when the trigger showed up it shipped the project's (dummy) credentials out. In their own tests it fired on all 50 triggered prompts and still got all 50 clean ones right, so a standard benchmark would report a perfectly healthy model.
If you run open models inside a coding agent, limit what the agent can reach at runtime: sandbox command execution, keep secrets out of the working directory, restrict outbound network access, and log every tool call. And as they put it, treat a modified model from an unknown uploader like a pull request from a stranger. 🔐 #MLEngineering
Florent BARTOCCIONI retweeted
We trained an LLM with FlashAttention-3 in BF16 🔥 Training looked healthy for a long time. Then the grad norm shot up 1000× 📈 and the loss ended 0.2 nats above FP32 attention. Not a single NaN 😶
The cause is in the attention backward 🧵
📄 arxiv.org/abs/2609.34272
Florent BARTOCCIONI retweeted
It’s easy to tell whether Muon was used to train a model just by looking at the empirical spectral distribution of its hidden layer weights. Compared with SGD/Adam, Muon leaves a very distinctive spectral signature in W^T W. This is because Muon updates low-magnitude directions more aggressively.
Florent BARTOCCIONI retweeted
What is the role of academic computer vision research in the age of increasingly powerful large models? Is GPT-6 Astra a step change? How can a researcher have an impact today in academia?
These are the questions I ask myself as I head off to ECCV 2026, a conference I’ve attended since 1992. One of my papers this year is VIGA, a method that takes an image as input and outputs a 3D Blender scene that represents that image. This is a classical inverse-graphics task and VIGA was the first method to solve it using an agentic approach.
The idea is now several years old and the first version of the paper was rejected. This delayed publication significantly. After it was accepted at ECCV, it was quickly surpassed by people using Claude Code for the same purpose. Today GPT-6 Astra blows away all previous results. But we still head off to ECCV to tell the community about our invention that is now fully out of date.
The way academic work often progresses is that one reads recent papers, notices that they have limitations, comes up with a new idea, explores this, publishes it, etc. Any published paper I read today is based on ideas that are at least a year old. And those ideas were based on the literature of the time, which was also a year old. That means that any paper I see at ECCV is likely two years out of date. In AI today, two years means your work is likely irrelevant.
At CVPR this summer I noticed that many authors have not gotten the message. They continue to work on “old” problems that have a long history. This history is based on assumptions about how the “vision problem” will be “solved”. The truth is that it is being solved in a very different way and many of these problems are no longer relevant. Another group of papers focuses on very niche problems where large models likely fail because of insufficient data or lack of business interest. The impactful papers were largely from industry and had long author lists and massive data+compute behind them. These papers were also out of data, describing systems that had been released months before, but at least they served to provide the community with more complete documentation and analysis of commercial systems.
So what should academics do? First, we need to put aside the tools we’ve used for years and start from scratch. Every project should start by trying really hard to solve the problem with existing tools. I would like to see every paper begin with a detailed experimental analysis of how existing models perform and why they fail (if they do). This gives the kind of insight we need today. Then, assuming current models fail, the solution should provide some fundamental insight that will outlive the next release of such models.
Reviewers today still focus on technical novelty. This pushes people to focus on tweaking architectures rather than clearly moving the field forward. Papers need to be judged based on their novel insight and not their novel technical contribution. This is a real shift in thinking but it focuses us on what matters - progress of the field.
If we want there to be a “field” of computer vision, then it can’t become a marginal backwater, focusing on esoteric problems. If you haven’t tried using Astra (or whatever comes next) to solve your problem, then you have not done your homework. This omission should be seen as negatively as not having a previous work section.
Concretely, I think papers should include a new section analogous to “Related Work” where that related work is current models and how they perform on the task. Reviewers should start asking for this and expecting authors to be able to articulate their insights about the limitations of existing large models.
I'm interested in your thoughts.
Florent BARTOCCIONI retweeted
When we say Atlas has pixel-perfect camera control, we mean it.
Atlas precisely follows input camera parameters, including non-planar projections such as the Brown-Conrady distortion model and the Kannala-Brandt fish-eye model.
@BhamidipatiPan1 invented a novel method of camera conditioning and it works beautifully.
🧵 [1/N]
Florent BARTOCCIONI retweeted
1/
Can Video World Models Track Unobserved World States?
Video world models increasingly look like simulators. But rendering plausible video is not the same as knowing what the world is doing.
We test this with a video shell game. A ball hides under one of five identical cups, which are swapped N times out of sight. Every frame is easy to render. The hidden arrangement is not.
A standard causal DiT trained on five swaps still renders ten swaps at 40.4 dB PSNR, but lifts the wrong cup. LaCT (TTT-KVB w. SwiGLU) tracks the correct final state with similar visual quality.
Florent BARTOCCIONI retweeted
Today we sharing Atlas, our new multimodal world model.
One reason this model is so special to me is that it combines two core visual intelligence tasks I have worked on for over a decade: generation and reconstruction
Florent BARTOCCIONI retweeted
Earlier this summer we let our intern connect his SO-101 to our excavator.
Intern at @ursamining if you’re an ambitious young engineer.
Florent BARTOCCIONI retweeted
Yo wtf CHILL, we released the robot less than 24h ago. This is too fast
Florent BARTOCCIONI retweeted
Behavioral cloning mystery
seohong.me/blog/behavioral-c…
I wrote a new blog post about "mysteries" in behavioral cloning that appear with real-world robot data (e.g., overfitting is "good"). I also tried to demystify them and shared my thoughts!
Florent BARTOCCIONI retweeted
pretty crazy how frontier models have almost no sense of time, they can't estimate how long a task will take or how long they've been working on it. they're also pretty bad at evaluating their own work
very nice blog with with clean experiments on this
lesswrong.com/posts/eAbuPXbj…
💥New blog post: Are LLM agents time-aware? Can they predict wall-clock time of tasks? Can they estimate time they spent?
We study this on a range of tasks, including long-horizon ones (ProgramBench, PaperBench, DeepSWE, etc).
Joint work with my MATS mentee @MOfengenden!
Florent BARTOCCIONI retweeted
You can have James Bond as your personal driver.
Just got this clip from a follower showing their @Tesla avoiding a collision recently on the highway while on FSD (Supervised) V14.
Here are all the angles. Self-Driving data at top:
6 of 9. Almost in place.
The operation began at 7:00 a.m. on Monday, 27 July, mobilizing close to 100 ITER staff members and contractors across two shifts. Once it lands, two-thirds of the ITER plasma chamber will be in place. #ITER #FusionEnergy #Tokamak #Engineering
Florent BARTOCCIONI retweeted
This is quite an interesting paper...
DEXA scans are used to measure bone density and body composition.
Here, authors trained a LeJEPA model on 11,540 DEXA scans which can then be used to extract clinically meaningful, whole-body representations of body.
For example, the model can predict Type 2 Diabetes, osteoporosis, biological age, and so much more.
Link to paper: arxiv.org/abs/2608.02208
Florent BARTOCCIONI retweeted
Meta AI researcher who created DINOv2 explains modern self-supervised learning!
Also introduces a new approach called CAPI
@TimDarcet gave a brilliant talk about his research at the @MedARC_AI journal club last month.
Very clear and information-dense, I learned a lot from his talk.
Give it a watch!
Florent BARTOCCIONI retweeted
Why do Vision-Language-Action (VLA) models still struggle with contact-rich manipulation? 🤖
Excited to share our new paper: Demystifying When and Why VLAs Fail in Contact-Rich Tasks and How to Fix Them. 🧵 (1/8)
Florent BARTOCCIONI retweeted
Can a Latent Action Model really know what the “action” is?
In Super Mario, Mario 👨🦰 moves but so do 🍄, ☁️,🌳, and the view🎥.
TL;DR: under agent ambiguity, don’t force LAMs to find the true action directly. We factorize what changed.
Paper: arxiv.org/abs/2606.30544v1


































