Building agents that reason, adapt, and act with RL and friends!
Joined December 2012
- Tweets637
- Following1.8K
- Followers839
- Likes1.7K
Pinned Tweet
Too many ideas die at the edge of the LLM/VLM/VLA training framework. Not anymore.
Excited to release FeynRL: built to help you understand, modify, and build new RL methods without fighting the stack.
github.com/FeynRL-project/Fe…
Try it, ⭐ it, and send feedback.
Exciting time to be at Modal 🚀 i'm hiring interns & full-time researchers in RL, ML, and large models to join our **new** research team. apply on our website or reach out to me directly (contact info on my website)!
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
Rasool Fakoor retweeted
RL is hot and has a reputation of being difficult to control. That is, unless you have the right parametrization. Off-policy RL, in particular struggles with the fact that your roll-out looks different from what you're actually optimizing. This can happen if you roll out with a more powerful model, or if you want to roll out in FP8, while you're training in FP16, or if you update asynchronously, or in other words, almost all the time in practice.
This is exactly why open-source frameworks like FeynRL matter more than ever. Open weights are NOT enough. Open AI needs open training recipes, alg, etc.
You should be able to understand training, modify it, & build new algorithms, optimizers, recipes.
github.com/FeynRL-project/Fe…
Rasool Fakoor retweeted
Higgs Audio v3 TTS is here.
Built for voice AI that speaks, not just reads:
• 100 languages with single-digit WER/CER
• inline control over emotion, style, prosody, and sound effects
• API, Workspace, and open weights
• Blog 👉 boson.ai/blog/higgs-audio-v3…
Watch the demo 👇
Off-policy data does not have to be a bug in RL. In our work, we shift the question from:
Is this data on-policy? -> How much should we trust this batch?
That change leads to a adaptive objective for RL LLM-training.
blog: feynrl-project.github.io
Rasool Fakoor retweeted
New blog post: Effective Sample Size
Reweighting data to fix distribution shift kills bias but piles everything onto a few points. ESS measures how much data you actually have left, the dial for when a replay buffer goes stale.
alex.smola.org/posts/40-effe…
Rasool Fakoor retweeted
Shoutout to my fantastic co-organizers for making the first-ever workshop on RL Environments & Agent Evals such a success!
@rasoolfa, @anishathalye, @aagohary, @natashajaques, @TheAndiPenguin, @migballesteros, Aziza Mirsaidova, Priyaranjan Pattnayak, Ahmed Elgohary, Alina Gavrilov, Aparna Elangovan, Graham Horwood
Rasool Fakoor retweeted
Packed room to hear @alexgshaw and @ryan_marten break down how @harborframework grew into *the* framework for RL environments.
In our RLEval workshop at @CAISconf today, attendees tackled big open challenges in RLEs & Agent Evals + I shared the approach we take at @joinHandshake
Join us tmrw if you are around at RLEval, first edition ever. Together with my wonderful co-organizers (@jomulr , @anishathalye, Alina Gavrilov, Aziza Mirsaidova) we have put together an exciting program with a great lineup of speakers and papers.
See the full program here:
rl-eval.github.io/
ps: I’ll also give a talk on what is wrong with current RL methods and frameworks, and why these issues can slow progress in using RL for large-model.
Tomorrow in San Jose: RLEval. Trillions going into LLM agents and we still cannot reliably evaluate them. 19 papers, talks from Alex Dimakis, Corby Rosset, Rasool Fakoor, and others. I'll be presenting Submodular Benchmark Selection from @boson_ai.
rl-eval.github.io/
Rasool Fakoor retweeted
Super useful resource, thank you for putting it together!
Researchers working on RLE design & Agent Evals might consider submitting papers / attending the first-ever Workshop in this area at the upcoming ACM Conference on AI and Agentic Systems:
rl-eval.github.io
One thing I keep hearing is that RL for L(L)Ms is "mostly a systems problem now" and the RL part is basically good enough.
I really don’t buy that. Current RL algs are still fragile as hell. Better systems help, but they don’t magically make the RL problem go away.
Rasool Fakoor retweeted
Our organizing team is excited for a productive day of discussion with you on May 26:
@natashajaques, @TheAndiPenguin, @rasoolfa, @anishathalye, @migballesteros, @aagohary,
Aziza Mirsaidova, Priyaranjan Pattnayak, Ahmed Elgohary, Alina Gavrilov, Aparna Elangovan, Graham Horwood
Are you working on RL, principled ways to build RL envs for agent training, or effective evaluation for agents? Want to showcase your NeurIPS submission? or just discuss about research more broadly?
Then consider submitting and attending to our first ever workshop on Methods and RL Environments for Evaluating AI Agents. Deadline: May 11
rl-eval.github.io
Want to build your own voice agent? 🎙️
Want to learn, have fun, and win prizes? 🏆
Then this hackathon is built for you! Join now and start creating. 🚀
At the Boson AI Higgs Audio Hackathon in collaboration with @Eigen_AI_Labs , you’ll work with ultra-low latency inference, expressive prosody modelling, advanced voice cloning + audio understanding with our model served by Eigen AI.
Apply today: luma.com/3vnw0e0q
