#CS Associate Prof @YorkUniversity #ComputerVision Researcher, @VectorInst Affiliate @UofT status only, @ELLISforEurope Member, X-social media chair 4all vision

Toronto, Ontario
Joined August 2015
Include

Only show posts containing:

Exclude

Hide posts containing:

Time range
-
Minimum likes
Replying to @FabianCabaH
Concept → Story → Storyboard → Script → Codex + Blender + After Effects → Iterate. Iterate. Iterate.
11
Kosta Derpanis retweeted
The Platonic Representation Hypothesis argues that vision and text are two projections of the same underlying world. As vision and language models scale, their representations converge toward this shared reality. We ask: Can physical robot motion be viewed as another projection of that same world? Our answer is yes. We find that more capable embodied models better align their vision and language representations with the structure of motion trajectories. 🚀 Based on this, we introduce Sensorimotor Alignment (S²): A Platonic Proxy for Embodied Policy Success. 📈 First, we find that S² is positively correlated with downstream robot task success across model families, in both simulated and real-world settings—making S² a training-free predictor of robot policy success. 🔍 Furthermore, we use S² to explain, at the representation level, several phenomena in VLAs and WAMs that were previously observed only through closed-loop results—such as why VLAs struggle to follow language instructions and why Fast-WAM can skip test-time future generation. 💡 We believe Sensorimotor Alignment offers a low-cost, representation-level view of robot foundation models that closed-loop success rates alone cannot reveal. More: 🌐 Blog & Video: yunfeixie233.github.io/s2/ 📄 Paper: arxiv.org/abs/2610.31670 💻 Code: github.com/yunfeixie233/sens… I’m sincerely grateful to Howard H. Qian, Shiyi Lan, Kaiyu Hang, and Chen Wei @Chen_Wei for their guidance and collaboration.
3
8
40
1,967
Kosta Derpanis retweeted
anthropic just killed the REA it stops claude from doing it that’s why local models are important
271
393
120
7,216
361,817
Replying to @jon_barron
New lecture material 🤓
2
429
Found one! Just arrived 🤗
RIP my favourite slide clicker. It’s been discontinued too 😢 Now what? 🤔
2
473
Kosta Derpanis retweeted
You can animate flowing water in splats now with more splats. And yes it also works with lava 😂
4
10
1
85
4,703
The Canadian in me 😉
11
Kosta Derpanis retweeted
We’re hiring a summer research intern at the MIT–IBM Computing Research Lab in Boston. We are looking for a PhD student near graduation. We’re interested in non-autoregressive architectures for language generation and reasoning, including diffusion models, flows, and EBMs.
2
8
1
144
6,075
Kosta Derpanis retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
876
2,613
1,435
19,717
4,446,446
Kosta Derpanis retweeted
Hello, #CVPR2027 World! We’ll be back with more coverage soon.
3
27
2,487
Kosta Derpanis retweeted
Ideogram Vector + Opus 5.5 = 🔥
Been experimenting with @ideogram_ai's new Vector model 👀 One prompt → a clean, editable SVG storyboard. Then passed it to @claudeai Opus 5.5 → a fully animated explainer about Brain Rot. 🧠 From vector design to motion graphics. #Ideogram #ClaudeAI #MotionDesign
2
1
18
1,961
Replying to @1rreverant
Ah, I see what you mean. The depth of field is blurring the separate trajectories together. I’ll fine-tune the focus so their motion reads more clearly. I do sweat the details. Thanks!
1
11
Replying to @1rreverant
I’ll talk to my agent and see what we can do 🤓
1
12
Replying to @1rreverant
As I said, work in progress 😉
11
Orbit are the level sets of the base Gaussian (debated on including it). The flow lines shift because the velocity field depends on the noise step t. As t changes, the predicted velocity at each point changes, so the streamlines change too. That’s expected in flow matching.
1
2
187
Replying to @1rreverant
It’s a trailer not an instructional video! 🤓 But this is v1, may revisit. Thanks!
1
2
171
Kosta Derpanis retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
50
241
38
2,392
132,171
Replying to @1rreverant
Trailer synopsis: Sample image noise → point in n-dimensional space → motion through an evolving vector field → image 🤷
1
30