Pieter-Jan Hoedt (@ml_hoedt) profile picture

Just another ML student with a twitter account

Austria
Joined September 2019
Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)Photo by Pieter-Jan Hoedt (@ml_hoedt)
In case you missed it: I am one of the contributors to the first ICLR blog post track this year! I (tried to) write about Normalizer-Free networks (openreview.net/forum?id=IX3N…) and what they can and can not tell us about normalization: iclr-blog-track.github.io/20…
1
12
2
32
Pieter-Jan Hoedt retweeted
Huge thanks to my co-first author @sir_luk for the great collaboration, and to all co-authors @thsschmied, @anamariarp17, @davidstap, @ml_hoedt, @maxmbeck and Sebastian Böck for their support and collaboration throughout this project. A special thank you to my PhD supervisors, @gklambauer and @HochreiterSepp , for their guidance and support. I also want to thank @nvidia for the outstanding open-source Nemotron datasets 💚, which were a major enabler for this project. We are equally grateful to the OLMo team @allen_ai for their open datasets and technical reports , which provided valuable insights throughout this work. Paper: arxiv.org/abs/2603.15590
3
7
637
Pieter-Jan Hoedt retweeted
xLSTM Distillation: arxiv.org/abs/2603.15590 Near-lossless distillation of quadratic Transformer LLMs into linear xLSTM architectures enables cost- and energy-efficient alternatives without sacrificing performance. xLSTM variants of instruction-tuned Llama, Qwen, & Olmo models.
5
57
8
307
25,010
Pieter-Jan Hoedt retweeted
We’re excited to introduce TiRex — a pre-trained time series forecasting model based on an xLSTM architecture.
5
22
6
69
14,925
Thrilled to announce two new developments at JKU and NXAI that are released today: - We scaled xLSTM to 7B parameters: linktr.ee/xlstm - For the people caring about state tracking capabilities, there's the new FlashRNN library: arxiv.org/abs/2412.07752
2
11
26
2,296
Super hyped to share NeuralDEM -- the first real-time simulation of industrial particulate flows. NeuralDEM replaces Discrete Element Method (DEM) routines and coupled (CFD-DEM) multiphysics simulations. 🧵 📜: arxiv.org/abs/2411.09678 🖥️: nx-ai.github.io/NeuralDEM/
7
91
8
387
41,302
Pieter-Jan Hoedt retweeted
Proud of our Geometry-Informed Neural Net (GINN🍸) which generates diverse shapes without training data. Paper: arxiv.org/abs/2402.14009 Great work and thanks to my amazing team mates and supervisors @artuursberzins @e_volkmann @SebSanokowski @HochreiterSepp @jo_brandstetter
Geometry-Informed Neural Networks are evolving! Beyond faster training and improved shapes, GINNs surprised us with an emergent property – a structured latent space. 🧵
3
1
17
1,882
Huge progress by @artuursberzins @AndyRadler @e_volkmann on Geometry-Informed Neural Networks (GINNs)! Faster training, better shapes, and surprising insights from enforcing diversity. 📜: arxiv.org/abs/2402.14009 🖥️: arturs-berzins.github.io/GIN…
We introduce Geometry-Informed Neural Networks to train shape generative models without any data (!!), combining learning under constraints, neural fields as a suitable representation, and generating diverse solutions to under-determined problems: 🖥️: arturs-berzins.github.io/GIN…
2
31
2
206
130,595
Pieter-Jan Hoedt retweeted
We are excited to introduce Bio-xLSTM! TLDR: we extend xLSTM to genomic, protein and molecular domains and find that it is a proficient generative model, learns rich representations and can perform in-context learning.
1
12
2
27
4,070
We have been messing around with xLSTMs on biological data. Turns out it works pretty well... Joint work with @smdrnks @Lisa_Schnecki @phseidl @JSchimunek @jo_brandstetter @AndreasMayr11 @sohvi_luukkonen @jo_brandstetter and @gklambauer
Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences xLSTM also shines for DNA, proteins and small molecules -- can handle large-range interactions and huge context! P: arxiv.org/abs/2411.04165
2
18
2,187
Pieter-Jan Hoedt retweeted
Transformers can be slow for real-time applications like robotics. We study if modern recurrent architectures, like xLSTM and Mamba, can be faster alternatives. Experiments on 432 tasks show that they compare favourably in terms of performance and speed 🎃 arxiv.org/abs/2410.22391
3
44
5
215
29,520
Pieter-Jan Hoedt retweeted
PAPER DEADLINE IS NOVEMBER 1st!!! ELLIS Machine Learning for Molecules Workshop 2024! Submit your papers here: openreview.net/group?id=ELLI… Workshop website: moleculediscovery.github.io/…
18
1
38
5,089
Relying on this formula to measure predictive uncertainty? You might measure the wrong thing, depending on your assumptions. Time to shed light on the basics of uncertainty estimation. 🧵👇
3
50
1
243
40,314
Interesting in scaling up neural operators? Happy to announce that Universal Physics Transformers (UPT) -- a scalable framework for neural operators is accepted at #neurips2024. Paper: arxiv.org/abs/2402.12365 Project page: ml-jku.github.io/UPT/
4
60
4
253
28,237
Pieter-Jan Hoedt retweeted
Am I the only one tired of LLM releases to gain 5% of accuracy? Time for audio, video, time-series, medical, biology, chemistry AI releases to get more of the spotlight please!
51
140
18
1,284
87,516
Pieter-Jan Hoedt retweeted
ELLIS Machine Learning for Molecules workshop December 6, 2024, HYBRID Paper submission deadline: November 1st. Program chairs: @fra_grisoni @jtmargraf an me Webpage: moleculediscovery.github.io/… We hope to see you all there!
2
33
1
80
6,188
Pieter-Jan Hoedt retweeted
MIM-Refiner: a self-supervised method to efficiently pre-train large-scale vision models, that sets a new SOTA on a broad range of datasets/tasks by refining masked image models (MIM, e.g. MAE or data2vec) with an instance discrimination objective (ID, e.g. SimCLR or DINO).
1
16
1
22
3,541
Pieter-Jan Hoedt retweeted
I've heard rumors that @ELLISforEurope ML4Molecules workshop will be back in December! Stay tuned for more updates! Here is a list of previous workshops: moleculediscovery.github.io/… moleculediscovery.github.io/… moleculediscovery.github.io/…
17
2
89
6,868
Pieter-Jan Hoedt retweeted
Austria's biggest Artificial Intelligence project "Bilateral AI" with 33M€ is about to start and we are looking for a PROJECT MANAGER Work together with @HochreiterSepp, @jo_brandstetter, and many more.. Project: bilateral-ai.net/ Apply: karriere.jku.at/hcm/jobexcha…
10
1
35
3,527
Pieter-Jan Hoedt retweeted
For those of you attending #ICML2024 , please consider joining our workshop on ML for Life and Material Science, July 26th Featured speakers/panelists include @mmbronstein and @HochreiterSepp ! ml4lms.bio/
1
11
1
73
7,592