Pinned Tweet
🎉 Excited to share that our paper, AP-OOD, has been accepted at @iclr_conf!
We introduce Attention Pooling for Out-of-Distribution Detection:
AP-OOD detects OOD inputs in the latent token space of Transformer models, drastically improving results on natural language tasks.📈
Claus retweeted
xLSTM Distillation: arxiv.org/abs/2603.15590
Near-lossless distillation of quadratic Transformer LLMs into linear xLSTM architectures enables cost- and energy-efficient alternatives without sacrificing performance.
xLSTM variants of instruction-tuned Llama, Qwen, & Olmo models.
📈Empirical results demonstrate that increasing the number of trainable weights in attention pooling consistently enhances OOD detection.
This suggests that further scaling may yield additional performance gains.
1
221
The paper and code are available here:
📜 arxiv.org/abs/2602.06031
💾 github.com/ml-jku/ap-ood
Many thanks to the great team 🙌: Christian Huber, Bernhard Lehner, @ido87, @HochreiterSepp, and @wzell7
1
194
Claus retweeted
🧪🧠 Do LLMs actually understand molecular structure?
📄 Paper: arxiv.org/pdf/2601.15279
🏆 Live leaderboard: huggingface.co/spaces/ml-jku…
Introducing MolecularIQ — a benchmark designed to test real molecular understanding (not memorization).
Claus retweeted
# AI in Drug discovery just BROKE THROUGH a wall #
A newer AI model, ConGLUDe, as fast but much more accurate than DrugCLIP.
Instead on just 40K structure-based data, ConGLUDe is trained on 100M datapoints from ligand-based data
P: arxiv.org/abs/2601.09693
BIG BREAKTHOUGH: A new AI tool could dramatically speed up the discovery of life saving medicines.
Researchers at Tsinghua University created a new system called DrugCLIP, that can screen drug molecules against human proteins at a speed that makes traditional methods look ancient.
> DrugCLIP uses deep contrastive learning to turn both molecules and protein binding pockets into vectors and match them almost instantly.
> It screened 500 million molecules across 10,000 human proteins, covering half of the entire human druggable proteome.
> The system completed 10 trillion molecule protein evaluations in a single day, roughly 10 million times faster than classic docking simulations.
> They used AlphaFold2 to generate protein structures and then refined binding pockets with a custom tool called GenPack.
> The model even identified compounds for TRIP12, a protein linked to cancer and autism that has resisted traditional drug-targeting approaches.
All data and models are open access, so labs worldwide can now speed up early stage drug discovery.
Claus retweeted
Ever experienced instabilities when using the popular LV (Log Variance) loss for training Diffusion Bridge Samplers?
Claus retweeted
Measuring AI Progress in Drug Discovery - A NEW LEADERBOARD IN TOWN
2015-2025: turns out that there's hardly any improvement. AI bubble?
GPT is at 70% for this task, whereas the best methods get close to 85%.
P: arxiv.org/abs/2511.14744
Leaderboard: huggingface.co/spaces/ml-jku…
Claus retweeted
🚀 Excited to share our new paper on scaling laws for xLSTMs vs. Transformers.
Key result: xLSTM models Pareto-dominate Transformers in cross-entropy loss.
- At fixed FLOP budgets → xLSTMs perform better
- At fixed validation loss → xLSTMs need fewer FLOPs
🧵 Details in thread
Claus retweeted
General relativity 🤝 neural fields
This simulation of a black hole is coming from our neural networks 🚀
We introduce Einstein Fields, a compact NN representation for 4D numerical relativity. EinFields are designed to handle the tensorial properties of GR and its derivatives.
Claus retweeted
We release AB-UPT, a novel method to scale neural surrogates to CFD meshes beyond 100 million of mesh cells. AB-UPT is extensively tested on the largest publicly available datasets.
📄 arxiv.org/abs/2502.09692
🤗 huggingface.co/EmmiAI/AB-UPT
💻 github.com/Emmi-AI/AB-UPT
Claus retweeted
Ever wondered how linear RNNs like #mLSTM (#xLSTM) or #Mamba can be extended to multiple dimensions?
Check out "pLSTM: parallelizable Linear Source Transition Mark networks". #pLSTM works on sequences, images, (directed acyclic) graphs.
Paper link: arxiv.org/abs/2506.11997
Claus retweeted
We’re excited to introduce TiRex — a pre-trained time series forecasting model based on an xLSTM architecture.
Claus retweeted
Happy to introduce 🔥LaM-SLidE🔥!
We show how trajectories of spatial dynamical systems can be modeled in latent space by
--> leveraging IDENTIFIERS.
📚Paper: arxiv.org/abs/2502.12128
💻Code: github.com/ml-jku/LaM-SLidE
📝Blog: ml-jku.github.io/LaM-SLidE/
1/n
Claus retweeted
Need to predict bioactivity 🧪 but only have limited data ❌?
Try our interactive app for prompting MHNfs — a state-of-the-art model for few-shot molecule–property prediction. No coding or training needed. 🚀
📄 Paper:
pubs.acs.org/doi/10.1021/acs…
🖥️ App:
huggingface.co/spaces/ml-jku…
Claus retweeted
1/11 Excited to present our latest work "Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics" at #ICLR2025 on Fri 25 Apr at 10 am!
#CombinatorialOptimization #StatisticalPhysics #DiffusionModels

























