
#CS Associate Prof @YorkUniversity #ComputerVision Researcher, @VectorInst Affiliate @UofT status only, @ELLISforEurope Member, X-social media chair 4all vision
Toronto, Ontario
Joined August 2015
- Tweets41.9K
- Following202
- Followers88K
- Likes93.1K
Replying to @FabianCabaH
Concept → Story → Storyboard → Script → Codex + Blender + After Effects → Iterate. Iterate. Iterate.
Kosta Derpanis retweeted
The Platonic Representation Hypothesis argues that vision and text are two projections of the same underlying world.
As vision and language models scale, their representations converge toward this shared reality.
We ask: Can physical robot motion be viewed as another projection of that same world?
Our answer is yes. We find that more capable embodied models better align their vision and language representations with the structure of motion trajectories.
🚀 Based on this, we introduce Sensorimotor Alignment (S²): A Platonic Proxy for Embodied Policy Success.
📈 First, we find that S² is positively correlated with downstream robot task success across model families, in both simulated and real-world settings—making S² a training-free predictor of robot policy success.
🔍 Furthermore, we use S² to explain, at the representation level, several phenomena in VLAs and WAMs that were previously observed only through closed-loop results—such as why VLAs struggle to follow language instructions and why Fast-WAM can skip test-time future generation.
💡 We believe Sensorimotor Alignment offers a low-cost, representation-level view of robot foundation models that closed-loop success rates alone cannot reveal.
More:
🌐 Blog & Video: yunfeixie233.github.io/s2/
📄 Paper: arxiv.org/abs/2610.31670
💻 Code: github.com/yunfeixie233/sens…
I’m sincerely grateful to Howard H. Qian, Shiyi Lan, Kaiyu Hang, and Chen Wei @Chen_Wei for their guidance and collaboration.
Kosta Derpanis retweeted
anthropic just killed the REA
it stops claude from doing it
that’s why local models are important
Kosta Derpanis retweeted
You can animate flowing water in splats now with more splats. And yes it also works with lava 😂
Kosta Derpanis retweeted
We’re hiring a summer research intern at the MIT–IBM Computing Research Lab in Boston.
We are looking for a PhD student near graduation.
We’re interested in non-autoregressive architectures for language generation and reasoning, including diffusion models, flows, and EBMs.
Kosta Derpanis retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com/
Kosta Derpanis retweeted
Ideogram Vector + Opus 5.5 = 🔥
Been experimenting with @ideogram_ai's new Vector model 👀
One prompt → a clean, editable SVG storyboard.
Then passed it to @claudeai Opus 5.5 → a fully animated explainer about Brain Rot. 🧠
From vector design to motion graphics.
#Ideogram #ClaudeAI #MotionDesign
Replying to @1rreverant
Ah, I see what you mean. The depth of field is blurring the separate trajectories together. I’ll fine-tune the focus so their motion reads more clearly. I do sweat the details. Thanks!
Replying to @1rreverant
I’ll talk to my agent and see what we can do 🤓
Replying to @CSProfKGD @1rreverant
Orbit are the level sets of the base Gaussian (debated on including it).
The flow lines shift because the velocity field depends on the noise step t. As t changes, the predicted velocity at each point changes, so the streamlines change too. That’s expected in flow matching.
Replying to @1rreverant
It’s a trailer not an instructional video! 🤓
But this is v1, may revisit. Thanks!
Kosta Derpanis retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
Replying to @1rreverant
Trailer synopsis: Sample image noise → point in n-dimensional space → motion through an evolving vector field → image 🤷













