Jake Schuster retweeted
I departed Google DeepMind after 8 years. So many fond memories—from early foundational papers in Google Brain (w/ @noamshazeer @ashvaswani @lukaszkaiser on Image Transformer, Tensor2Tensor, Mesh TensorFlow) to lead Gemini posttraining evals to catch up & launch in 100 days, then leading the team to leapfrog to LMArena #1 (and stay there for over a year!), and finally working on the incredible reasoning innovations for Gemini’s IMO & ICPC gold medals (w/ @HengTze @quocleix).
Gemini has been a wild journey from one paradigm to another: first, revamping our LaMDA model (the first instruction-like chatbot!) from an actual chatbot to long contentful responses with RLHF; then, reasoning and deep thinking by training over long thinking chains, novel environments, and reward heads. When we first started, public sentiment was bad. Everyone thought Google was doomed to fail due to its search legacy and organizational politics. Now, Gemini is consistently #1 in user preference and spearheading new scientific accomplishments, and everyone thinks Google winning is obvious. 😂 (It also used to be the case that OpenAI would jump the AI newscycle by announcing before us from a backlog of ideas for every new Google release; safe to say that backlog is empty.)
I have since joined xAI. The recipe is well-known. Compute, data, and O(100) brilliant, hard-working people are all that’s needed to obtain a frontier-level LLM. xAI *really* believes in this. For compute, even at Google I have never experienced this # of chips per capita (& 100K+ GB200/300K’s are incoming with Colossus 2). For data, Grok 4 made the biggest bet in scaling RL & posttraining. xAI is making new bets to scale data, deep thinking, and the training recipe. And the team is quick. No company has gotten to where xAI is today in AI capabilities in as little as time. As @elonmusk says, a company’s first- and second-order derivatives are the most important: xAI’s acceleration is the highest.
I’m excited to announce that in my first few weeks, we launched Grok 4 Fast. Grok 4 is an amazing reasoning model, still the top on ARC-AGI and new benchmarks like FinSearchComp. But it’s slow and was never really targeted for general-purpose user needs. Grok 4 Fast is the best mini-class model—on LMArena, it is #8 (Gemini 2.5 Flash is #18!), and on core reasoning evals like AIME, it is on par with Grok 4 while 15x cheaper. S/o to @LiTianleli @jinyilll @ag_i_2211 @s_tworkowski @keirp1 @yuhu_ai_
Jake Schuster retweeted
companies like Facebook record every imaginable interaction their users have with the platform. they log each of your clicks and taps. they keep track of how long your gaze lingered on a post, whether you were on the same WiFi as that woman who might be your friend, which instagram reel you watched three times.
for a single user this is quaint, but these practices are done on a planetary scale across all technology giants. they create petabytes of data per day and keep it for as long as the European regulators will let them. then they can have machine intelligence instrument it into useful knowledge for their cybernetic control systems that build newsfeeds, serve ads, decide how much compute to spend on you, which SKUs should be in which warehouses right before you want them. the Hive metastore bills run into the billions
hospitals throw most of their data and telemetry out after each case, every single day. they record videos of vascular surgeries, endoscopies, discovering interesting physiologies. sometimes they're not recorded at all and most of them the time they delete them as soon as they’re done
it's even worse for physiologic waveforms (ECG, EEG, arterial lines) which are essentially never recorded anywhere at all. milisecond scale views of patient's brains, vasculatures, hearts are generated and instantly destroyed. all of these time series of course predict people's hearts stopping, brains exploding, etc ahead of time. surgeons teleoperate robots, none of the micro-movements are recorded, policies never learned, never correlated into which outcomes were successful or not
this would be unthinkable to most software people whose instinct is to record everything everywhere never mind the cloud costs, because we are sure there will be some use for it later and some model to be trained later. i don't have a prescription here per se my point is just that our civilization routinely hoards and treasures some of the silliest data in the world "i pressed like on the john pork reel" & destroys much of all the most important data it generates and limits what machines can learn
Jake Schuster retweeted
The energy provisions in the Republicans' One Big Horrible Bill are truly so bad! Who wants this? The country's automakers don't want it. Electric utilities don't want it. Data center developers don't want it. Manufacturers in energy intensive industries don't want it. It straight up murders the boom in battery, solar and wind manufacturing investment. It jacks up your utility bill & raises prices at the pump. It kills a half a trillion dollars of pending investment in US manufacturing and energy supply and 100s of thousands of associated jobs. It makes our air dirtier and our climate more dangerous.
Seriously: Who the fuck benefits from this, besides Joe Craft and his coal mining interests and a couple of big gas producers? Republicans are screwing over their constituents, American business interests, and our country's future in an epic way here.
Jake Schuster retweeted
Palantir cofounder Peter Thiel 🔮
“The next Mark Zuckerberg won’t be building a social networking site. The next Bill Gates won’t build an operating system.”
“You don’t want to be the Harvard of North Dakota. The something of somewhere is often the nothing of nowhere.”
Via @HooverInst, 2014
Jake Schuster retweeted
.@NVIDIA selects #Navitas! NVIDIA collaborates with Navitas on their next-gen 800 V HVDC architecture to support 1 MW IT racks and beyond!
Read today’s news, here: bit.ly/45jhwaG
$NVTS $NVDA #NVIDIA #Semiconductor #Technology #AI #DataCenter #Hyperscale #Computex
Jake Schuster retweeted
I'm sick of all the software slop. Millions of lines, and 90% of them do not express any ideas. They simply exist to support the other millions of lines.
Dead weight abstraction layers that are too costly to refactor away. Plumbing linking old systems to newer systems to the new system that everyone has to move to. But we can't really delete the old thing it still handles this legacy use case and that legacy code isn't tested nearly well enough to try and refactor it.
tinygrad is trying to be different. 13067 lines of pure Python, including a torch-like frontend, a gradient library, a scheduler, a symbolic algebra library, an MLIR replacement, LLVM like rewrite passes, and support for 16 backends, including a full driver for AMD GPUs that mmaps PCIe BARs.
Each line expresses an idea. There is no boilerplate. And if there is, the code base is small enough we can still refactor it away. 13k lines is something you can read in a weekend. Unlike with million line codebases, it's still possible to fit the entire thing in your head (or in an LLM context window). And because that's possible, it's still possible to refactor.
If you are sick of all the slop of modern software, come work on tinygrad. We strive toward the platonic ideal of a Tensor library.
Jake Schuster retweeted
Learning to Drive from a World Model blog.comma.ai/mlsim
Jake Schuster retweeted
Reinforcement Learning in the era of LLMs requires scalable, distributed systems to push the boundaries of reasoning and alignment.
Today - we release Atropos - our RL environments framework.
github.com/NousResearch/Atro…
Atropos is a rollout framework for reinforcement learning with foundation models that supports complex and diverse environments for advancing the capabilities of foundation models.
In Greek mythology, Atropos was the eldest of the three Fates. While her sisters spun and measured the threads of mortal lives, Atropos alone held the shears that would cut these threads, determining the final destiny of each soul. Just as Atropos guided souls to their ultimate fate, this system guides language models toward their optimal potential through reinforcement learning.
The work on Atropos was led by @dmayhem93 and built alongside @teknium, @rogershijin, @max_paperclips, @nullvaluetensor, @JSupa15, @artemsya and @karan4d
Jake Schuster retweeted
HOLY CRAP, a new super tiny 1.6B param voice model just dropped that seems to.. outperform 11labs!? 😵💫
From Nari-labs, Dia is an Apache 2.0 voice model, that can generate laughs, sniffs and emotions, copy an existing voice and is effectively real time on larger GPUs:
Jake Schuster retweeted
Your incumbent has ego, many competing priorities, businesses to avoid cannibalizing, a product that does too many things, rules that were invented for a different time ago, to avoid angering old users w a new UX, too many employees to convince.
That’s why you can win.
Jake Schuster retweeted
I'm going to call it right now. A lot of stuff is going to break on this mission.
By design.
As part of the plan.
Don't get upset. I'm not saying SpaceX plans to fail. I'm pointing out that SpaceX has taken an ultraimportant principle from software engineering, and realized it applies to all engineering.
Feedback beats planning.
And that, you see, is why SpaceX doesn't do things the NASA way. The NASA way was to gold-plate everything, plan and test and plan and test, and generate mountains of paper detailing every contingency, with every scenario prepared for.
SpaceX just shrugs, says "it's unmanned", and sends it.
Half the time it blows up. That's the whole point. They don't actually want it to blow up, of course, but they're anticipating that it might.
That possibility is part of the plan. Because one rocket blowing up, or crashing, in an actual end-to-end test, beats many, many man-years of planning and plotting.
The key realization here is that knowledge only comes from empirical observation. Everything else is just speculative.
The sooner you get into a feedback loop, and the faster you run it, the more iterations you can do in less time. This means, while others are planning and speculating, you actually learn something.
Relevant data is the most precious thing in the universe. And it's worth blowing up any number of rockets to get it.
Because rockets are just stuff. They're just made of stuff. And you can always get more stuff.
You can never get more time.
So expect to see a lot of things go wrong on this, and other SpaceX missions. Anticipate it. Accept it when it happens. Doesn't mean the dream of the stars is dead.
It just means we're doing it cowboy style.
This is a valuable lesson for our own lives. If there's something you want to do, something you want to try, some goal you have, it's easy to dip a toe in the water, test the temperature, and plan. A lot.
Planning makes us feel good if we're afraid. Because it provides us with the illusion of security. Never mind that we don't know which scenarios are actually going to happen, never mind that we're planning for the wrong thing, planning makes us feel safe. And if we're nervous, we can plan forever.
But the difference between the expert and the novice isn't theory or intelligence or plans. It's relevant domain knowledge. Gathered from empirical observation.
So the trick is to get into that feedback loop as soon as possible, and run it as fast as possible. Give yourself the most possible opportunities to learn, per unit time.
We only learn while we are moving.
Jake Schuster retweeted
EasyControl just dropped on Hugging Face
Adding Efficient and Flexible Control for Diffusion Transformer
Free chatgpt style ghibli image generation with easy control
Jake Schuster retweeted
College admissions are unjust.
Not just biased. Not just broken. Unjust.
Students spend high school anxiously stacking their résumés with hollow activities, then collect generic recommendation letters and outsource their essays to tutors or AI. Admissions at elite colleges now come down to who you know, your identity group, or how well you play the game.
This system rewards manipulation, not merit. It selects for conformity, not character.
That’s why we’re introducing the University of Austin’s new admissions policy:
If you score 1460+ on the SAT, 33+ on the ACT, or 105+ on the CLT, you will be automatically admitted, pending basic eligibility and an integrity check. Below that threshold, you’ll be evaluated on your test scores, AP/IB results, and three verifiable achievements, each described in a single sentence.
That’s it.
We care about two things: Intelligence and courage.
Intelligence to succeed in a rigorous intellectual environment (we don't inflate grades). Courage to join the first ranks of our truth-oriented university.
College admission should be earned—not inherited, bought, or gamed. At UATX, your merit earns you a place—and full tuition scholarship.
Apply here: uaustin.org/admissions
Jake Schuster retweeted
I’ve changed so little. From my 1978 Bachelor’s thesis:
“The adult human mind is very complex, but the question remains open whether the learning processes that constructed it in interaction with the environment are similarly complex. Much evidence and many peoples’ intuitions suggest that the learning processes are in fact simple and that the adult mind’s complexity is due to a long history of adaptive interaction with a complex environment.”
Jake Schuster retweeted
Nvidia just released GR00T-N1-2B on Hugging Face
open foundation model for generalized humanoid robot reasoning and skills
Jake Schuster retweeted
A banger just got released 💥
Here is a snapshot of L2D, the biggest self-driving dataset by far!
- 90 TeraBytes of data
- 5000 hours of driving
- 6 surrounding HD cameras
- OPENLY AVAILABLE
- Train your car to drive like @Tesla at home
🧵 More details in thread
Jake Schuster retweeted
Replying to @skirano
Sutton & Silver had seen it a few years ago in “Reward is enough” sciencedirect.com/science/ar…
Jake Schuster retweeted
Highly non-convex toy optimization problem.
SGD vs Adam vs Evolutionary Strategies (ES).
Let's gooooooo...



























