@tk_rusch

Faculty @ELLISInst_tue and @MPI_IS leading the Computational Applied Mathematics & AI Lab (CAMAIL) + Scientific Advisor @liquidai || Prev. @MIT_CSAIL + @ETH_en

 Cambridge, MA
camail.org
Joined May 2020
  • Tweets272
  • Following301
  • Followers1.1K
33Media
Konstantin Rusch retweeted
Vera Milovanović@nevera_m2h
I learned a lot from this paper, thanks to the authors! 😍Building on FPRM’s fixed-point approach, QRR introduces rotational symmetry to recognize convergence even while the hidden state keeps moving ---> it stops in fewer iterations!
Sumin Park@psmizizOct 15
Can reasoning finish while latent states keep moving? Happy to share Quotient Recurrent Reasoner (QRR)! We let symmetry-generated rotations persist while task-relevant dynamics converge faster in quotient space, reaching 97.9% on Sudoku-Extreme with 78% fewer loops than FPRM.
1
7
314
This is by far the coolest visualization I’ve seen on here in a while!! 🔥👇
Sumin Park@psmizizOct 15
Can reasoning finish while latent states keep moving? Happy to share Quotient Recurrent Reasoner (QRR)! We let symmetry-generated rotations persist while task-relevant dynamics converge faster in quotient space, reaching 97.9% on Sudoku-Extreme with 78% fewer loops than FPRM.
1
7
673
Konstantin Rusch retweeted
Liquid AI
@liquidaiOct 15
Today we release 𝚍𝟷-𝚞𝚕𝚝𝚛𝚊, our most powerful decision model. > built for hard decision problems. > d1-ultra is multimodal > supports 44 languages > #1 on the @ArtificialAnlys Decision Index (65) > leads all eight categories of our 112-evaluation suite 1/
66
189
85
1,606
296,881
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
Can reasoning finish while latent states keep moving? Happy to share Quotient Recurrent Reasoner (QRR)! We let symmetry-generated rotations persist while task-relevant dynamics converge faster in quotient space, reaching 97.9% on Sudoku-Extreme with 78% fewer loops than FPRM.
2
38
3
290
16,876
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
Our paper, Looped Reasoning Finishes Earlier Than You Think, is now on arXiv, with code available below! Paper: arxiv.org/abs/2610.18174 Code: github.com/camail-official/Q… With my advisor T. Konstantin Rusch (@tk_rusch). We'd love to hear your thoughts! 6/🧵

Looped Reasoning Finishes Earlier Than You Think

Looped reasoning with dynamic depth can be viewed as relaxation toward solution attractors in a learned dynamical landscape. In practice, however, latent trajectories often oscillate, cycle, or...

arxiv.org
3
19
695
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
QRR combines faster convergence with stable reasoning performance. With 7M parameters, QRR reaches: 97.9% on Sudoku-Extreme 85.8% on Maze-Hard / 97.0% on Maze-Unique 62.4% Pass@1 on Mini-ARC 5/🧵
1
1
1
6
665
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
On Sudoku-Extreme, QRR reaches solution roots much sooner than FPRM (median 122 vs 234 lfwd). It also halts the reasoning loop earlier (27 vs 122 recurrent steps), with only 0.1% false halts, compared with 8.9% for FPRM. 4/🧵
1
1
7
518
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
FPRM and HRM often develop a dominant slow oscillatory mode that delays convergence. QRR's learned quotient dynamics, in contrast, settle earlier without a leading slow-mode bottleneck. Removing the leading mode provides only a small further gain in QRR unlike others. 3/🧵
1
1
5
562
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
A dominant oscillatory mode can impose its slow decay rate on coupled latent dynamics. We predict its coupling effect from local Jacobian analysis, then attenuate that mode once in FPRM and HRM. The measured responses and settling-time gain match our predictions. 2/🧵
1
1
8
624
Konstantin Rusch retweeted
Sumin Park@psmizizOct 15
FPRM shows alternating flips while HRM follows spirals near solution roots. QRR's raw state can also rotate while its quotient trajectory contracts. The key distinction is whether these oscillations affect the remaining computation, even as raw motion continues. 1/🧵
1
1
15
1,147
Introducing Quotient Recurrent Reasoners (QRR) 🚀 Looped reasoning models (e.g., TRM, FPRM) spend a large amount of loops on oscillatory dynamics that don't contribute to reasoning, wasting a lot of computation before converging. Turns out gauge equivariance helps tremendously by defining convergence in a quotient space instead of state-space, where these oscillations are part of the same equivalence class. Super strong performance on reasoning benchmarks: Sudoku-extreme: 97.9% 🤯 Incredibly work done by my student Sumin Park @psmiziz Check out the fascinating thread below. Also such a cool visualization! 🧵👇 @ELLISInst_Tue @MPI_IS @liquidai
Sumin Park@psmizizOct 15
Can reasoning finish while latent states keep moving? Happy to share Quotient Recurrent Reasoner (QRR)! We let symmetry-generated rotations persist while task-relevant dynamics converge faster in quotient space, reaching 97.9% on Sudoku-Extreme with 78% fewer loops than FPRM.
3
24
4
179
7,141
Konstantin Rusch retweeted
Eric
@Ex0bytOct 15
fixed-size matrix-valued memory is all the rage! code: github.com/davidgonmar/depth… Paper: arxiv.org/pdf/2610.18809
David González-Martínez@_davidgonmarOct 14
Replying to @_davidgonmar
Joint work with @philna00 (co-first author), @tk_rusch and @Shiwei_Liu66. Paper: arxiv.org/abs/2610.18809 Code: github.com/davidgonmar/depth… Checkpoints: huggingface.co/collections/d…

Depth Memory

Residual connections propagate information through a network by additively accumulating layer outputs. Recently proposed alternatives, such as Manifold-Constrained Hyper-Connections (mHC) and...

arxiv.org
2
33
1,967
Konstantin Rusch retweeted
Songlin Yang
@SonglinYang4Oct 14
MesaNet rotated 90 degree?
David González-Martínez@_davidgonmarOct 14
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
1
15
114
7,255
Konstantin Rusch retweeted
Yifan Zhang
@yifanzhang_Oct 14
Very interesting work! Seems closely related to Deep Delta Learning. arxiv.org/abs/2601.00417

Deep Delta Learning

Transformer residual streams are updated by addition. A sufficiently expressive residual block can represent content replacement, but the residual update itself has no operation that reads,...

arxiv.org
David González-Martínez@_davidgonmarOct 14
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
2
9
108
6,715
Konstantin Rusch retweeted
Neehal Tumma
@ntumm120Oct 14
Super interesting work applying test-time regression along the depth axis
David González-Martínez@_davidgonmarOct 14
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
2
1
14
695
Konstantin Rusch retweeted
Philipp Nazari@philna00Oct 14
Replying to @_davidgonmar
🚀Also make sure to check out our blog post! phnazari.github.io/blog/2026…
2
10
637
Konstantin Rusch retweeted
Oscar Davis
@osclsdOct 14
Continuous Diffusion LMs went from niche to contender this year, but the recipe is shaky: why self-conditioning? How to embed tokens? 🤨 Our new work takes your one-hot flow from 4% to 50% on TinyGSM without self-conditioning, and explores the design space of CDLMs. 🔥🚀 (1/6)
13
52
6
200
15,940
Konstantin Rusch retweeted
Yu Zhang 🐙🌘@yzhang_csOct 14
delta rule rotated by 90°
David González-Martínez@_davidgonmarOct 14
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
4
14
2
151
11,066
Konstantin Rusch retweeted
flyingpetals@flyingpetal472Oct 14
Replying to @yzhang_cs
Significant work. And I think the memory stream may have potential on controlling the recurrent depth of the model.
2
4
369
Konstantin Rusch retweeted
Philipp Nazari@philna00Oct 14
Our new preprint Depth Memory is finally live! We propose a framework that unifies residual methods such as mHC and AttnRes and propose Linear Depth Memory, a novel depth mixer. Check it out! Interactive blog post: phnazari.github.io/blog/2026…
David González-Martínez@_davidgonmarOct 14
mHC and AttnRes demonstrate that richer depth mixing improves model performance at scale. Is there a unifying perspective? 🤔 Meet our Depth Memory framework: each layer reads from a set of streams to form its input, then writes back its output. Linear Depth Memory (LDM) equips models with a matrix-valued memory.
1
7
26
1,420
Load more