Happy to share what I’ve been working on since joining Genesis! GENE-26.5 is a one-of-a-kind, robotics-native multimodal foundation model that learns from diverse, in-the-wild data across modalities and outputs actions enabling a 54-DoF robot system to perform the most dexterous, long-horizon manipulation tasks to date—approaching human-level capability.
This is the result of innovations across the full stack—data collection and processing, robot systems, model architecture, training strategies, and scalable evaluation infrastructure.
Jensen keeps winning
NVIDIA has just announced a huge investment package into French startups.
@nvidia has now committed nearly $10 billion to the European tech ecosystem.
The French startups it is investing in include:
> Doctolib
> Entalpic
> Genesis AI
> Pigment
The company had already invested in other French companies such as Alice & Bob, Gradium and SCINTIL Photonics.
This is amazing news for the French and European ecosystem.
LETS GO
i love writing. i never let agents write for me. whether it's docs, or tweets, or shitposts, or even prompts (okay maybe that one's harder as agents manage other agents)
there's a simple reason for that: i still strongly believe that writing is thinking. it's a skill i hope to never give up. it's also important to me that whatever i say is always authentic, mistakes and all. my favorite posts are the ones that i just full send without any editing
i suspect it also helps with prompting. being able to clearly express what i want, and knowing the vocabulary i need to be able to direct my agents does seem to make for more effective usage of agents. in the same way that engineers have an advantage when we prompt agents to write code, because we are intimately familiar with the words that make up the discipline
knowing the right word incantations to get an agent to produce a desired goal almost feels like spellcasting. as models get more capable, and that input box looks more and more like a magic box that can output anything, words become the interface between you and the genie
there may come a time when we give up the keyboard in exchange for brain-computer interfaces. but until then, i'm writing every word.
Zu retweeted
Perhaps counterintuitive, but this is fundamentally true.
IK is built on a simple model of the arm and doesn’t account for things like friction, arm dynamics, environment. With RL you can get reactivity, distill to choose the right IK branches to avoid singularities.
Bitter lesson much?
Replying to @xiaolonw @chris_j_paxton
Any online optimization (IK, ID, MPC) for low-level control should be abandoned. A RL policy can fundamentally track more precisely, more robustly, and adaptively to unknown disturbances and uncharacterized noise, especially for low-cost hardware, even on a manipulator.
Very well said!
I’ll take the opposite point, that for the vast majority of people, it probably doesn’t make sense to do a PhD in AI right now, and is better to join industry. I say this with love for Phillip, whose work and philosophy I do admire, and respect for putting this out there!
Everything in AI is, generally speaking, working well. Most things come from stacking wins and good ideas over and over again, and if you’re in industry, you have a much better opportunity to learn what is actually working and what doesn’t work, and then push the forefront of what’s possible at a much larger scale. Internships also don’t really feel the same because you’re certainly not working on the most important projects for the company and often come in with a high risk agenda and work alone. In industry, you learn a lot about how to manage budgets, work in large teams, and push projects that require a lot of effort and work to get done.
Academic project budgets are generally quite minuscule, maybe being O($1 - $100K) for many projects and you’re often working alone. If you really want to push novel ideas, another route I don’t think most people consider is also a startup, where you may be able to raise 1-100x more money to do the same project you may do in a PhD, with more resources and the ability to hire other people. And, if it goes well, the ability to scale it dramatically.
When it comes to hiring, often times, it’s easier to get an offer having worked well with other people in industry and then having them change jobs and bring you over. Doing a PhD can get your foot in the door, but often it’s still for entry level positions after doing a 5-year PhD. In industry too, you also will make a lot of strong connections, similar to those you’d make in academia. Though it might be the case that it’s good to do 1 year of a PhD and then drop out to work in industry at this point to get the benefit of both.
Doing a PhD right now is on extremely hard mode, where not much work ends up mattering and even if it works, there’s high cost to change the ways of industry to adapt something that might provide incremental gains when everything generally is working well with enough resourcing and effort.
ICYMI: General-purpose robots won’t be solved by AI models alone. The full stack has to work together.
At #Actuate26, @gs_ai_ Co-Founder @theo_gervet shared why solving robotics means co-designing models, data, simulation, hardware, and control as one system.
Full talk: youtube.com/watch?v=LFHW50ju…
Zu retweeted
This makes sense given that NYU has the max venn intersection of top Math and top AI people.
@thegautamkamath any “interesting” comments made during this talk?
John Schulman (co-founder of Thinking Machines and OpenAI) wrote that goal-driven research beats idea-driven research.
I realized it's also really good life advice.
Don't chase what other people are doing. Figure out what life you want, and then figure out how to get there.
arxiv.org/pdf/2305.10455
Three years since the first draft. This is aging like fine wine 🍷
Zu retweeted
unfortunately i contacted them on the night of wednesday, september 2nd making the following points
1. i had intel they had information about math work i was doing and a few days before they had spun up a group to try to compete.
2. i emphasized and reemphasized mine was a strictly personal mathematical collaboration
3. that this was a collaboration that had been going on for a year
4. that in my personal collaboration we were happy users of codex as well as obviously claude (and i later complimented astra explicitly after it came out for its help, and reassured i would obviously be sprinkling acknowledgments for it, previous models, and codex liberally in the writeups)
5. that it would look terrible if openai were competing against mathematician consumers, and my collaborator (Tristan) wanted to email about this explicitly
6. that i expected none but in the worst case intended to refuse marketing around the result we were working towards, being the fruit of a personal external academic collaboration, and wanted to express this because i figured worries about such marketing by their leadership was why a team was spun up to extraordinarily directly compete.
sebastien and i have in the past months had our fun pushing each other on twitter, but as i process it all i keep feeling bad about Tristan.
1. neither of us could sleep Sunday night, so we went on a walk around the city. i finally asked him why fluids, and he told me it was the navier-stokes problem. he told me parts of the story about his classic nonuniqueness result with Vicol i hadn’t heard yet, and also about how he too grew up writing shaders as a kid and then switched to math
2. tristan is a mathematician’s mathematician. he kept rock solid to his principles during the tense calls and negotiations, immediately turning down a career dream and million dollar prize for me, the sort of strange excited unclear-quasiuseful confused idiot in our collaboration. fifteen minutes later, his feet were up on his desk with three toes per foot peeking out of ripped socks
3. our periodic call after i’d texted him euler blowup was just laughing for an hour or two. just laughing. i’d seen fable deal correctly with monge ampere in the equality case of ehrhart’s conjecture, so in some call before that i’d really pushed him for the craziest shit he could think of that i might try. i texted him the pdf as an update on that “diego luis 3” line, in which following his incredible expression of intuition i hodgepodge slapped together a bunch of our recent work, in particular correcting a bunch of better and lesser known papers in the literature, as well as on diego and luis’ ipm paper, and hoped an analogous construction for boussinesq (thus axisym euler) might work out. i managed to wade through enough confusion to insist to the model it stop telling me stupid regularity stuff about kelvin waves, and a rayleigh-taylor ansatz stuck. like i said, just laughing. when i texted him the pdf, the worst thing either of us had seen so far, i didn’t even comment on the euler blowup claim, only explaining why skepticism should remain for boussinesq, because i found it so shocking. i mean what the hell did i know, i must’ve confused myself by misunderstanding one adjective like i was used to with algebraic geometry. im so lucky to have been able to live that
4. he managed all this this entire time with unclear sleep, timestamps continue to amaze me, due to just recently having a kid
it is obvious he deserves the same sort of praise he heaped upon luis, perhaps mutatis mutandis:)
Replying to @__alpoge__
nothing at all was locked we were just willing to talk but you didn't talk to us ... sorry but that's just untrue, we were willing to go above and beyond and have as many discussions as you would have liked to reach resolution.
Zu retweeted
Recently, there have been a lot of impressive demos of AI agents, like Claude, controlling robots.
I wrote a short blog post with my thoughts on the advent of these "robot-use agents."
web.mit.edu/phillipi/www/wri…
I think it's an important change in the trajectory of robotics!
Very nice write-up! The skill becomes even more important when working on big projects with lots of people!
Btw, 🍩 is such a great name for the blog series! Eagerly waiting to read more from Toru!
Congrats @micoolcho @chris_j_paxton @DJiafei and thanks for having me as the #3 guest! Was fun chatting about sim-to-real dexterous manipulation 😄
Amazing to see how much the AI robotics community has grown since then. Look forward to the next 100 episodes!
We've hit 100 episodes!
Here's a look back on our journey so far (website with some stats):
robopapers100.com/
Some highlights in the thread 🧵:
Zu retweeted
Today we sharing Atlas, our new multimodal world model.
One reason this model is so special to me is that it combines two core visual intelligence tasks I have worked on for over a decade: generation and reconstruction




























