Post-training LLMs at @Aleph__Alpha | Ex intern at @Nvidia and @Bosch_AI
Santa Clara
Joined May 2023
- Tweets84
- Following199
- Followers209
- Likes352
Pinned Tweet
📢 #scholarGPT now can perform complex filtering operations like selecting a specific year 📅, making research even more efficient! 💪 Build with self-query from @langchain
Sign up on the waitlist to be the first to try it out mailchi.mp/d8b3dbb881ae/scho…! 🚀✨ #ChatGPT #Research🧑💻
Leonard Salewski retweeted
During its development, Kolibri surprised us in ways we didn't expect with reward hacking & escaping sandboxes. We were a little proud but also increased our sandbox isolation
For an 80B-A3B model, it is unexpected. All the details are in "Lessons learned" section in tech report
Leonard Salewski retweeted
We started the year with the conviction that Europe needs its own sovereign AI. Built in Germany from scratch and on infrastructure in Europe specifically for our customers.
This weekend, we released Kolibri.
Meet the team behind it 🧵
Leonard Salewski retweeted
My favorite reinforcement learning plot from our Kolibri report: Inference v.s. training log probs, background is the logit gradient norm of importance-sampled Reinforce. Numerically unstable for small inference log-probs, but you can do exact token-level gradient clipping in RL.
Here are two neat speed tricks we found while SFT'ing Kolibri 🧵
1: We use standard Next-k-fit online bin packing with document masking maximises useful tokens per sequence. Whats easy to miss: less padding means a larger effective batchsize, so the learning rate can go up.
In post-training, we taught Kolibri to think and answer in both English and German consistently across SFT and RL alike. We generated large amounts of synthetic data to control how Kolibri answers and trained with RL on more than 1.2 million tasks, including ones built around our customer needs, and ones to teach it to say “I don’t know” when the context is insufficient.
This adds a small bias: a step's microbatches are no longer a plain random draw. In our runs the wall-clock gain won by a wide margin: median throughput per GPU went from 8,610 to 15,850 tokens/s on 64 B300s, +84%, same data, model and kernels.
Check section 3.1.4.1 of our tech report (aleph-alpha.com/downloads/te…) for more details. It also has many more details, so certainly worth a read on how to train a model end-2-end!
Here is one interesting detail from our post-training:
Kolibri scores above 90% on AIME 2025 and AIME 2026 just after SFT. We also evaluated on an internal AIME 2026-like eval with new IID samples and got even better results. For math, RL was just the finishing touch than.
In post-training, we taught Kolibri to think and answer in both English and German consistently across SFT and RL alike. We generated large amounts of synthetic data to control how Kolibri answers and trained with RL on more than 1.2 million tasks, including ones built around our customer needs, and ones to teach it to say “I don’t know” when the context is insufficient.
Leonard Salewski retweeted
Big congrats to my friends at Aleph. Glad I was still around when this team formed and got to see how much work went into building it end to end.
kolibri.constructlabs.com
We will offer inference for free for the next ~48 hours.
Hit me up if you are interested in bringing this into production or post-training it.
Leonard Salewski retweeted
Dear @Aleph__Alpha team - thank you for making Kolibri-1 open.
We care deeply about sovereign AI, and launching it on German Unity Day makes today feel especially fitting.
As a small gesture of support, we’ve hosted and made Kolibri-1 free for anyone to try for the next few days.
No GPU. No setup. Just try it.
tesseracted.com/kolibri-1-ch…
Open Models move us all forward.
We’re rooting for you. 🇩🇪
Tesseracted Labs GmbH.
tesseracted.com
#alephalpha #openweightmodels
Almost all models today have reasoning effort, here is how we built it for Kolibri. Our model supports four efforts: none, low, medium, high.
However, our SFT data had no effort labels. Here is how we got them anyway, using only existing data 🧵
RL then trains on all three efforts and separates them further with a length-dependent format signal.
This is a first draft and there is amazing research out there - we have much more planned for future model iterations, stay tuned.
All of this was a team effort with the amazing folks at @Aleph__Alpha! Check out the tech report for more details: aleph-alpha.com/downloads/te… or download the weights from HF: huggingface.co/Aleph-Alpha/K….