Pinned Tweet
In case you missed it: I am one of the contributors to the first ICLR blog post track this year! I (tried to) write about Normalizer-Free networks (openreview.net/forum?id=IX3N…) and what they can and can not tell us about normalization:
iclr-blog-track.github.io/20…
Pieter-Jan Hoedt retweeted
Huge thanks to my co-first author @sir_luk for the great collaboration, and to all co-authors @thsschmied, @anamariarp17, @davidstap, @ml_hoedt, @maxmbeck and Sebastian Böck for their support and collaboration throughout this project. A special thank you to my PhD supervisors, @gklambauer and @HochreiterSepp , for their guidance and support.
I also want to thank @nvidia for the outstanding open-source Nemotron datasets 💚, which were a major enabler for this project. We are equally grateful to the OLMo team @allen_ai for their open datasets and technical reports , which provided valuable insights throughout this work.
Paper: arxiv.org/abs/2603.15590
Pieter-Jan Hoedt retweeted
xLSTM Distillation: arxiv.org/abs/2603.15590
Near-lossless distillation of quadratic Transformer LLMs into linear xLSTM architectures enables cost- and energy-efficient alternatives without sacrificing performance.
xLSTM variants of instruction-tuned Llama, Qwen, & Olmo models.
Pieter-Jan Hoedt retweeted
We’re excited to introduce TiRex — a pre-trained time series forecasting model based on an xLSTM architecture.
Pieter-Jan Hoedt retweeted
Thrilled to announce two new developments at JKU and NXAI that are released today:
- We scaled xLSTM to 7B parameters: linktr.ee/xlstm
- For the people caring about state tracking capabilities, there's the new FlashRNN library: arxiv.org/abs/2412.07752
Pieter-Jan Hoedt retweeted
Super hyped to share NeuralDEM -- the first real-time simulation of industrial particulate flows. NeuralDEM replaces Discrete Element Method (DEM) routines and coupled (CFD-DEM) multiphysics simulations. 🧵
📜: arxiv.org/abs/2411.09678
🖥️: nx-ai.github.io/NeuralDEM/
Pieter-Jan Hoedt retweeted
Proud of our Geometry-Informed Neural Net (GINN🍸) which generates diverse shapes without training data.
Paper: arxiv.org/abs/2402.14009
Great work and thanks to my amazing team mates and supervisors @artuursberzins @e_volkmann @SebSanokowski @HochreiterSepp @jo_brandstetter
Pieter-Jan Hoedt retweeted
Huge progress by @artuursberzins @AndyRadler @e_volkmann on Geometry-Informed Neural Networks (GINNs)! Faster training, better shapes, and surprising insights from enforcing diversity.
📜: arxiv.org/abs/2402.14009
🖥️: arturs-berzins.github.io/GIN…
We introduce Geometry-Informed Neural Networks to train shape generative models
without any data (!!), combining learning under constraints, neural fields as a suitable representation, and generating diverse solutions to under-determined problems:
🖥️: arturs-berzins.github.io/GIN…
Pieter-Jan Hoedt retweeted
We are excited to introduce Bio-xLSTM! TLDR: we extend xLSTM to genomic, protein and molecular domains and find that it is a proficient generative model, learns rich representations and can perform in-context learning.
We have been messing around with xLSTMs on biological data. Turns out it works pretty well...
Joint work with @smdrnks @Lisa_Schnecki @phseidl @JSchimunek @jo_brandstetter @AndreasMayr11 @sohvi_luukkonen @jo_brandstetter and @gklambauer
Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences
xLSTM also shines for DNA, proteins and small molecules -- can handle large-range interactions and huge context!
P: arxiv.org/abs/2411.04165
Pieter-Jan Hoedt retweeted
Transformers can be slow for real-time applications like robotics. We study if modern recurrent architectures, like xLSTM and Mamba, can be faster alternatives. Experiments on 432 tasks show that they compare favourably in terms of performance and speed 🎃 arxiv.org/abs/2410.22391
Pieter-Jan Hoedt retweeted
PAPER DEADLINE IS NOVEMBER 1st!!!
ELLIS Machine Learning for Molecules Workshop 2024!
Submit your papers here: openreview.net/group?id=ELLI…
Workshop website: moleculediscovery.github.io/…
Pieter-Jan Hoedt retweeted
Relying on this formula to measure predictive uncertainty? You might measure the wrong thing, depending on your assumptions. Time to shed light on the basics of uncertainty estimation. 🧵👇
Pieter-Jan Hoedt retweeted
Interesting in scaling up neural operators? Happy to announce that Universal Physics Transformers (UPT) -- a scalable framework for neural operators is accepted at #neurips2024.
Paper: arxiv.org/abs/2402.12365
Project page: ml-jku.github.io/UPT/
Pieter-Jan Hoedt retweeted
Am I the only one tired of LLM releases to gain 5% of accuracy? Time for audio, video, time-series, medical, biology, chemistry AI releases to get more of the spotlight please!
Pieter-Jan Hoedt retweeted
Hey there 👋
Want to do a Phd with me? Apply: it-u.at/en/study-program/doc…
Pieter-Jan Hoedt retweeted
ELLIS Machine Learning for Molecules workshop
December 6, 2024, HYBRID
Paper submission deadline: November 1st.
Program chairs: @fra_grisoni @jtmargraf an me
Webpage: moleculediscovery.github.io/…
We hope to see you all there!
Pieter-Jan Hoedt retweeted
MIM-Refiner: a self-supervised method to efficiently pre-train large-scale vision models, that sets a new SOTA on a broad range of datasets/tasks by refining masked image models (MIM, e.g. MAE or data2vec) with an instance discrimination objective (ID, e.g. SimCLR or DINO).
Pieter-Jan Hoedt retweeted
I've heard rumors that @ELLISforEurope ML4Molecules workshop will be back in December!
Stay tuned for more updates!
Here is a list of previous workshops:
moleculediscovery.github.io/…
moleculediscovery.github.io/…
moleculediscovery.github.io/…
Pieter-Jan Hoedt retweeted
Austria's biggest Artificial Intelligence project "Bilateral AI" with 33M€ is about to start and we are looking for a PROJECT MANAGER
Work together with @HochreiterSepp, @jo_brandstetter, and many more..
Project: bilateral-ai.net/
Apply: karriere.jku.at/hcm/jobexcha…
Pieter-Jan Hoedt retweeted
For those of you attending #ICML2024 , please consider joining our workshop on
ML for Life and Material Science, July 26th
Featured speakers/panelists include @mmbronstein and @HochreiterSepp !
ml4lms.bio/






























