Alexander Theus retweeted
1/ New preprint: GRASS-MIL, a tool for discovering which tumor architectures matter.
From spatial single-cell data it maps the tissue neighborhoods tied to a clinical outcome, gives each an exact, signed contribution, and groups them into recurring niches. 🧵
Self-supervised transcriptomic FMs are pretrained on millions of gene-expression profiles, yet often fail to beat simple supervised baselines on clinical tasks.
Why?
Our new work suggests the pretraining objective may matter more than scale.
📄arxiv.org/abs/2610.08694
1/8
We also extend GeneICL to right-censored survival prediction, without survival-specific pretraining, using a reduction based on Cox residuals.
Overall, our results suggest that in transcriptomics, how we pretrain may matter more than how large the model is.
8/8
Really enjoyed working on this with Michael Bohl, @val_boeva, and David Wissel!
Alexander Theus retweeted
Our Workshop on Weight-Space Symmetries @icmlconf @weightsymmetry is starting now! You can reach Room 403 by going up from Hall A.
Program: weightsymmetry.com/schedule
👇 Check out our new preprint: FPRM (Fixed-Point Reasoning Model).
• No ACT head: loop to a fixed point, with convergence as the halt signal.
• No hierarchy: a single loop beats HRM/TRM variants!
• SOTA among 7M models: Sudoku-Extreme 94.2%, Maze-Hard 87.0%, ARC-AGI-1 47.5%.
Alexander Theus retweeted
This TRM variant makes a transformer block a contractive map, so that looping becomes a fixed-point process. They leverage this by approximating gradients with Neumann series (Truncated BPTT). Very cool work!
🔗arxiv.org/abs/2606.18206
Alexander Theus retweeted
#ICLR2026
Into mode connectivity, model merging, or permutation invariance? We show how optimization dynamics shape the loss landscape of merged weights. Come check it out!
📅 23/04 10:30AM – 13:00PM
📍 Pavilion 3 P3-1809
w/ @TheusResearch @DamienTeney @orvieto_antonio
Alexander Theus retweeted
📢 Submissions are OPEN for the Weight Space Symmetry Workshop @icmlconf!
⏰ Deadline extended → April 30 (23:59 AOE)
Consider submitting any work related to weight symmetries: optimization, model merging, weight space learning, and so on!
#ICML2026 #weightsymmetry2026
Excited to announce that our paper has been accepted as an Oral at NeurIPS 2025! 🥳
1/ 🚨 New paper alert! 🚨
We explore a key question in deep learning:
Can independently trained Transformers be linearly connected in weight space — without a loss barrier?
Yes — if you uncover their rich symmetries.
📄 arXiv: arxiv.org/abs/2506.22712
1/ 🚨 New paper alert! 🚨
We explore a key question in deep learning:
Can independently trained Transformers be linearly connected in weight space — without a loss barrier?
Yes — if you uncover their rich symmetries.
📄 arXiv: arxiv.org/abs/2506.22712
9/ 🔑 Takeaway:
Transformers can be linearly connected — but only if you exploit richer network symmetries.
We show that general symmetry alignment (not just permutations) unlocks low-loss paths across ViTs and GPT-2.
10/ 📄 Paper: arxiv.org/abs/2506.22712
By: @Theus__A , Alessandro Cabodi, @SAnagnostidis , @orvieto_antonio , @unregularized , and @val_boeva
🙏 Huge thanks to my amazing co-authors for this collaboration!
#Transformers #LMC #MachineLearning #DeepLearning
