Previously Compute scaling @OpenAI, Co-founder at Moonshine AI, @CerebrasSystems, @googleai, @nvidia

Mountain View, CA
Joined July 2007
Scale out is a pyramid scheme. The human brain is not scaled out.
1
63
If RSI is going to be tail-recursive, I have a certain optimization in mind.
1
59
Shouldn't it be called Iterative Self Improvement? I mean, where is the Recursion?
1
2
270
Tokens are plenty. Tokens are smart. But how will humans use them? With the help of other humans. Hence no job loss. Simple as.
239
What did roon see?
not a day to lose now
249
Systolic arrays at HBM densities. Stop dragging bytes across the chip. Put the matrix engines where the bandwidth is.
178
Better to try and fail than to not try at all, and spend forever wondering.
102
I want someone more opinionated than me to write the Misguided Managers Hypothesis to refute the Mismanaged Geniuses Hypothesis.
1
121
Premium token economics: Cost of a token = cost of energy. With 1 kWh, you can sip 1 kW for an hour, or guzzle 4 kW and get the same output in 15 minutes. Same tokens. Different speed. That speed? That’s what you’re paying a premium for. ⚡
1
103
Gearing up for something new. Knowledge forged over decades in the trenches, taste refined through thousands of failed experiments. Plus coding models. Let’s see what this concoction can create.
118
This
The Jensen + @dwarkesh_sp podcast was fantastic. Jensen is someone who understood how ecosystems work and someone who understands real-world trade, policy and controls work. And in some deeper sense how AI will actually diffuse into the world. In this podcast, Dwarkesh came off as someone who picked up talking points from an AGI party in the SF Mission District. And the contrast was so evident. As someone who understood ecosystems relatively deepy, maybe I understood Jensen's take more than others did (idk). Mythos, that Dwarkesh kept bringing up, is not a single absolute turning point in the AI development landscape. Take a state-of-the-art Chinese open-source model, and give it three orders of magnitude more test-time compute + post-training algorithmic advances that haven't been published yet. That's the baseline. It was evident that in whatever bubble Dwarkesh is in, that is seen as a naive or illogical baseline. When AI has such a complex development cycle, it's evident that America needs many levers of policy intervention across multiple layers in a dominant ecosystem that ideally the Western world controls. The entire premise that a particular model with AI development will have a critical phase change is neither correct nor does evidence point to it. OpenAI made this point with GPT-4, Anthropic made this point with Mythos, but neither stood / will stand the test of time. I think Jensen's repeated emphasis within the podcast to try to make this point mostly didn't get Dwarkesh's attention. And Dwarkesh (in this podcast) represents an entire cult of AI researchers and decision-makers that are going to influence policy. The thing with policy interventions is that if you do too much too early, you shoot yourself in the foot. There's a good reason American foreign policy and general sanctions of all kinds are measured and continuous. Despite Jensen's attempt at educating the "Anthro" audience how ecosystems work, I'm also not super hopeful a lot of people who've taken the extreme position will change their thought after listening to this podcast. I do think there's a certain religiousness that has permeated some of that community that would make it hard to understand ecosystems at a deeper level.
235
Being ghosted is the worst
2
313
Whoever gets the GPUs wins, not because they’re better, but because no one else was allowed to compete. It’s a closed loop of insider access, preferential deals, and early hoarding.
1
107
tfw you realize all the cool things you have learnt and done could have been much more impactful in a different setting but now are going to be lost forever. like tears in rain.
1
114
AI production vs AI diffusion Which activity is going to be more valuable in the next 5 years?
95
tfw you tipped alacrity past its edge into pushiness relearning the art of patience
98
AI proper vs. AI adjacent. If you are doing a startup you need to be aware which of these lanes you are on. It's technology risk vs. execution risk.
82
Coding agents have been great answer machines. But I have been the better question machine. Like way better. So far.
67
TFW when you cite an author in a lot of your recent papers and you are about to meet them imminently.
1
2
256
HF audio team member here 👋🤗 Don’t want to be the party pooper here, but those look a little… questionable 🙊 Would love to be proven wrong though, @WillowVoiceAI what about adding the model to the leaderboard? BTW We’re working on private test sets for the Open ASR Leaderboard to address this tipe of questions, but here, the model is the closest you can get (understandable ofc since your product is built on it)
Replying to @WillowVoiceAI
Most models score 5-7% word error rate on clean audio. In real-world conditions they fall to 10-15%. Atlas 1 holds at 1.2% on clean audio and 2.1% in production. The gap widens in noisy environments.
5
2
1
107
15,457