Pre-training LLMs @OVHaiLLM in Paris
Montpellier, France
Joined January 2020
- Tweets648
- Following340
- Followers1.3K
- Likes59.4K
Alexandre TL retweeted
Just finished reading The Little Book of RL by @AlexandreTL2.
Super clear, compressed walkthrough of the underlying math paired alongside minimal code. Great entry point for building core RL intuitions!
Excited to dive into my own RL experiments next!
The book, along with its supplementary material, is available for free here: github.com/alxndrTL/little-b…
Or you can buy a paperback copy on Amazon by looking up its name (sold at cost)
It is a much revised version of a video series I gave in 2020 here: youtube.com/@alexandretl/cou…
Alexandre TL retweeted
We’ve been lucky enough to test Mamba-3 ahead of the curve. 🧪
Here is how it integrates into Hybrid Models (Spoiler: it unlocks Muon for SSMs for the first time). 🧵
The newest model in the Mamba series is finally here 🐍
Hybrid models have become increasingly popular, raising the importance of designing the next generation of linear models.
We've introduced several SSM-centric ideas to significantly increase Mamba-2's modeling capabilities without compromising on speed. The resulting Mamba-3 model has noticeable performance gains over the most popular previous linear models (such as Mamba-2 and Gated DeltaNet) at all sizes.
This is the first Mamba that was student led: all credit to @aakash_lahoti @kevinyli_ @_berlinchen @caitWW9, and of course @tri_dao!
Alexandre TL retweeted
btw, i wrote a post about "how to scale" based on what i've learned over the past few months. it covers muP, HP scaling laws, and some stuffs. would be happy to get any feedback or discussion.
(it's pretty verbose and no TL;DR, sorry lol)
howtoscalenn.github.io
Alexandre TL retweeted
more fun open-source research news
- new paper drops (nGPT)
- claims 4-20x training speedup over GPT
- shocking
- very cool
- very valuable
- community tries to reproduce
- doesn't hold up
- turns out baseline was busted
- another cool new research idea oneshotted by github anon
Bcs the baseline by the looks of it was wrong github.com/NVIDIA/ngpt/issue…
Alexandre TL retweeted
magnet:?xt=urn:btih:5546272da9065eddeb6fcd7ffddeef5b75be79a7&dn=mixtral-8x7b-32kseqlen&tr=udp%3A%2F%2Fopentracker.i2p.rocks%3A6969%2Fannounce&tr=http%3A%2F%https://xitter.cf/t.co/g0m9cEUz0T%3A80%2Fannounce
RELEASE a6bbd9affe0c2725c1b7410d66833e24
Alexandre TL retweeted
Voici la 5ème vidéo de notre série pédagogique sur #ChatGPT pour les collégiens, lycéens, profs, parents 🦜🧑🎓🗣️ #IA
Ici nous expliquons comment il est possible de simuler un "raisonnement" ⚙️🤔avec ChatGPT, et comment celui-ci peut dérailler 🤖😅🤯
youtube.com/watch?v=uLMqTPmv…
Alexandre TL retweeted
🚀📢L'épisode 5 de notre série "#ChatGPT en 5mn" est sorti !🎓
Ici nous expliquons certaines forces 💪des modèles de langage🦜: prise en compte du contexte, utilisation d'outils, ou base de brainstorming 📚🧠💡!
#college #lycee #edTech #IA #education
youtube.com/watch?v=5HVR3cVF…
Dans les collèges et lycées tout le monde parle de #ChatGPT 🦜🧑🎓🗣️
Mais combien savent comment ça fonctionne ?⚙️ Quelles sont ses possibilités et ses limites ? 🤔
Pour les enseignants et les élèves, voici un tour d'horizon en 5 mn 🚀🤖💡 #education
youtube.com/watch?v=K8gOvC8g…
Alexandre TL retweeted
Can #LLMs help children learn in classrooms? 🧒🏼🎓
💡Yes! Super happy to share this field study wth #GPT helping 9-10 yo to learn asking curiosity-driven questions!🔍🤔
Amazing work led by @RaniaABDBenAmar @FlowersINRIA @T_EvidenceB #education #edTech
📖arxiv.org/abs/2211.14228
Alexandre TL retweeted
Here’s a great video from @AlexandreTL2 presenting our latest work on grounding #LLMs using online RL 📹
+ explanations of @ClementRomac on 5x acceleration and 5x lower memory footprint usin LoRA 👍💪
In our last @icmlconf paper, we study how LLMs can be aligned to external enviroment dynamics through online RL.
🎬 (by @AlexandreTL2 ):
youtube.com/watch?v=QB2jJ6D3…
In the past weeks, we've worked on improving the compute efficiency of our GLAM method 🚀
Here's what we learned 👇
Alexandre TL retweeted
In our last @icmlconf paper, we study how LLMs can be aligned to external enviroment dynamics through online RL.
🎬 (by @AlexandreTL2 ):
youtube.com/watch?v=QB2jJ6D3…
In the past weeks, we've worked on improving the compute efficiency of our GLAM method 🚀
Here's what we learned 👇
Alexandre TL retweeted
We've been working on a series of educational videos explaining Large Language Models in 5 minutes at @FlowersINRIA 🔍🤖
The first videos (in French) are now available on YT. English versions are coming soon!! 🤗
Dans les collèges et lycées tout le monde parle de #ChatGPT 🦜🧑🎓🗣️
Mais combien savent comment ça fonctionne ?⚙️ Quelles sont ses possibilités et ses limites ? 🤔
Pour les enseignants et les élèves, voici un tour d'horizon en 5 mn 🚀🤖💡 #education
youtube.com/watch?v=K8gOvC8g…
Alexandre TL retweeted
Et voici le 2ème épisode de notre mini-série sur les modèles de langage📽️🤖🆕
👉Le prompting, ou l’art de se faire comprendre par #ChatGPT: Explication en 5mn📝⏱️🦜
(par une équipe de recherche en #IA @FlowersINRIA)
#education #mediation #lycées @inria
youtube.com/watch?v=8IQ9i_Qo…
Alexandre TL retweeted
Dans les collèges et lycées tout le monde parle de #ChatGPT 🦜🧑🎓🗣️
Mais combien savent comment ça fonctionne ?⚙️ Quelles sont ses possibilités et ses limites ? 🤔
Pour les enseignants et les élèves, voici un tour d'horizon en 5 mn 🚀🤖💡 #education
youtube.com/watch?v=K8gOvC8g…
Alexandre TL retweeted
🚨🚨
Large Language Models encode world knowledge that can benefit RL agents when solving decision-making problems (e.g. in robotics 🤖).
But what happens when their knowledge are wrong? 🎯
Can we align LLMs and ground their knowledge through interactions with online RL? 🕹
Alexandre TL retweeted
Today we’re announcing that Facebook AI has built and open-sourced Blender, the largest-ever open-domain chatbot. It outperforms others in terms of engagement and also feels more human, according to human evaluators. ai.facebook.com/blog/state-o…
Alexandre TL retweeted
For students and others interested in expanding their knowledge of AI during this period, we thought it might be helpful to ask our researchers what they consider to be the most impactful and insightful resources available to use #AtHomeWithAI (1/9)