Josh Robb (@josh_robb) header
Josh Robb (@josh_robb) profile picture

operator, code nanny, product builder, unicorn baiter, @tendnz

Auckland, New Zealand
Joined June 2007
Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)Photo by Josh Robb (@josh_robb)
This quote from @WarrenBuffett made my day. Thanks @CBinsights
1
11
I see how some people can watch flashy robot demos on the timeline and think with a shiver that the robot apocalypse is around the corner, but reality check: There is no robot in the world that can change the batteries in a jack-o-lantern. If you have a robot friendly door handle it might be able to walk outside, pick it up, and carry it back inside. The robot could open the drawer with your batteries in it, but then things get tough. Opening the package of AA batteries? Extracting a precision screwdriver from its case? Unscrewing four tiny screws that require a hard press and twist? Popping batteries out of the case? Yes, you could build and train a specialist robot to do that today, but there is a very long way to go to replace the full range of human capabilities. Near term doomer scenarios should stop at “secretly controlling humanity” instead of “wiping humanity out” 😀
734
417
111
7,765
1,294,743
I cannot let a steam engine analogy go by without bringing up John Wilkinson, perhaps the most underappreciated engineer of the Industrial Revolution. Watt had figured out how to dramatically improve the steam engine, but he spent years struggling to actually manufacture one. The problem was embarrassingly simple: nobody could make a sufficiently round cylinder. The casting and machining tolerances were atrocious. Watt complained that one 18-inch cylinder was out of round by as much as THREE-EIGHTHS OF AN INCH. Steam leaked past the piston. He tried packing the gaps with cork, oiled rags, paper, and apparently even old hats. Nothing worked particularly well. Enter John Wilkinson, a British ironmaster whose immediate concern was manufacturing better cannons. At the time, iron cannons were commonly cast around a core, introducing defects and misalignment that could make the barrels inaccurate or dangerously weak. Wilkinson developed a process for boring cannons from solid castings, and in 1774 patented a vastly improved boring machine. The key innovation was almost comically mechanical: a massive, rigid boring bar, supported at both ends, that could cut a genuinely straight, circular hole instead of following the irregularities of the casting. He adapted this technology to Watt's cylinders. By 1776, Matthew Boulton was boasting that Wilkinson had bored a FIFTY-INCH cylinder accurate to within the thickness of an old shilling. Suddenly, the steam engine was a commercially viable machine. And the story gets better. Wilkinson became one of Watt's first customers, buying an engine to drive the blast machinery at his own ironworks. The steam engine was now helping manufacture the iron that made more steam engines possible. I find this whole episode absolutely extraordinary. The Industrial Revolution was, for a critical period, bottlenecked on the ability to make a sufficiently round hole. We remember Watt, and rightly so. We remember the invention of the steam engine and the subsequent development of thermodynamics. But the man who made the machinery that made the whole thing possible is practically a footnote. I think we consistently underestimate how much of scientific progress is actually downstream of advances in tooling, measurement, and manufacturing precision. It's not just that technology precedes theory. Sometimes the decisive breakthrough is that someone finally figures out how to build the thing the theorists have been talking about for twenty years. Anyway, I think John Wilkinson deserves considerably more credit for modern civilization than he gets.
a few thoughts: 1. pretraining science is kind of like how people built quite good steam engines before they understood thermodynamics. “Science owes more to the steam engine than the steam engine owes to science.”-(L.J. Henderson). if nick is right, which i am pretty sure he is, progress points in the direction of more interpretability, and pretraining was "slouching towards monosemanticity". my guess would be that major breakthroughs in interp also advance capabilities 2. nick is a good friend of mine, safe to say my origin story at openai wouldn't have happened if i hadn't met him, and it's fascinating watching him operate and seeing the early drafts evolve into this. he's the most profoundly happy guy I know- not driven by guilt, activism, or religious terror, but by fascination. he of course understands the stakes of understanding neural networks, but it doesn't seem like the core reason he does it: he just loves the neural nets. afaict he's posting this from some beautiful hideaway in tokyo. there's a lesson in there for all the +EV maxers. i feel very grateful to have met him 3. i do think it's crazy how few people across the world were staring at neural nets growing: most have moved to a different level of behaviorist analysis, thinking about reward functions and test scores and whatnot. it does not bode that well for the major model companies, oai included, that fundamental breakthroughs like this are coming from outside. we've become too much of a pressure cooker focusing on local incentives, studying narrow tracks under old paradigms, mostly doing engineering, rather than fundamental research, and implicitly hoping the next model will figure it out
68
395
44
3,662
101,797
Josh Robb retweeted
when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even eliezer will have to lower his pdoom. we can start making alignment into a 'normal science'
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
164
395
84
6,627
437,450
You can actually see the very early prototype-signs of this with the integer multiplication community auto researcher stuff that @RohanArun is coordinating. xitter.cf/rohanarun/status/21100…
The ideal outcome for science isn't a couple of labs using proprietary models and millions of dollars to crack trophy problems, it's to have millions of AI-aided human scientists who follow their curiosity and become way more ambitious
2
5
215
Josh Robb retweeted
(1/6) From @GoogleDeepmind’s AlphaGo to AlphaZero, we saw machines find moves that reached beyond what we’d imagined for them. No one taught them those strategies. They emerged from reinforcement learning: playing, failing, and trying again under a single binary task reward for success or failure. Inspired by them, we couldn’t help but wonder: what might a robot discover if we gave it that same freedom to learn? That question led us to AlphaNIST. Starting tabula rasa, with that same binary reward plus only a simple control cost, a single policy solves all 20 @NIST assembly tasks at a 98.2% success rate in simulation. More surprising still, AlphaNIST can complete 10 different tasks in a row, receiving that reward only once, after all 10 are done. Just as AlphaGo’s strategies surprised its creators, AlphaNIST discovered dexterous strategies we couldn’t have come up with ourselves. Paper: arxiv.org/abs/2610.13354 Website: alpha-nist.github.io
64
337
123
2,684
420,914
Humans provide a fun comparison point here. In the 80s, the "cognitive consequences of programming" was a major psychological research focus. The prevailing hypothesis was that learning formal logic and algorithmic structures through computer programming would act as a form of mental gymnastics that could upgrade general reasoning, planning, and novel problem-solving abilities. But foundational research, followed by subsequent meta-analyses, demonstrated that this does not happen. Students who learn programming become highly proficient at algorithmic thinking and coding, but these gains do not transfer to general cognitive tasks or unrelated problem-solving scenarios. Similarly, intensive mathematical training improves mathematical deduction and structural mapping, but does not increase an individual's baseline rate of skill acquisition in unrelated fields. General intelligence seems to be a fundamental property of the brain rather than a skill you can train. Practicing in a domain makes you better at the domain but does not make you generally smarter.
What if the jagged frontier is mainly math + code (which you can push arbitrarily far with RLVR), and everything else starts to plateau because it is still bottlenecked by human generated data? Model performance in non-verifiable areas has kept improving steadily, albeit much slower than for math and code. But is that steady improvement a side effect of a higher G (itself driven by RLVR), or only a function of the amount of new human data getting injected into training (which is still continually happening on a massive scale)? A lot of things depend on the answer to this question
93
136
27
1,345
74,644
Since the beginning @dhh has been a fan of meta programming. Agents a just a more powerful tool for codegen. Least surprising thing ever.
So basically @DHH came to the Rails World and told them that Ruby on Rails is no longer needed now because AI can make apps in any languages. That's the funniest thing that could've happened.
1
5
198
Josh Robb retweeted
So basically @DHH came to the Rails World and told them that Ruby on Rails is no longer needed now because AI can make apps in any languages. That's the funniest thing that could've happened.
Clearly, this @DHH Rails World 2026 keynote has become an instant classic. youtube.com/watch?v=V9SxpJpH… Highlights: - WebApps are dead since it is so cheap, easy, and fast to build native apps with agentic coding. - No need for a framework like Ruby on Rails if you can achieve the same functionality with e.g. Rust on the backend while saving a lot of money by reducing the number of needed servers and having a faster application. - Your application needs a CLI. Because agents love command-line tools.
13
45
7
926
77,568
Steve Kerr says Roger Federer taught his Warriors team the secret to sustained success wasn’t outworking everyone, it was building a life around your craft you never want to escape from “I’ll tell you a great story about Roger Federer. We were playing in China with the Warriors in 2017. Roger was in Shanghai for the Masters tournament, so we invited him to come speak to the team.” “Draymond Green asked him, ‘How do you sustain success? How have you managed to win majors 20 years after you first won one?’ Everyone was expecting him to say, ‘I work harder than everybody,’ the will, all the clichés.” “Roger Federer says, ‘I get up every morning and I make breakfast for my kids. Then I take them to school and drop them off. Then I go practice tennis for about two hours, and I’ve figured out a really good routine where I don’t destroy my body, but I can get all the work in I need.” “‘Then I go have lunch with my wife. In the evening, we cook dinner together. The kids are all home from school. We ask them about school. There’s so much joy in the house. We put them to bed, and then I put my head on the pillow and go, “Man, that was just a great day.” And I’ve been doing that for 20 years.’” “And it was like, yes. That’s it. That’s the formula. That’s what leads to sustained success. He loves tennis. He loves his family. He loves life. It’s not outworking everybody and banging your head against the wall, ‘I’m going to be better than everybody.’ It’s allowing your natural talent to shine through with a work ethic, with a great family life, with perspective, with peace, with mindfulness.” “It’s perspective. It’s joy. It’s passion. It’s mindfulness. It’s an awareness that we are truly lucky. To get the most out of ourselves, it’s our daily rituals and experiences and love and joy.”
221
3,272
313
23,901
2,565,166
Josh Robb retweeted
My LLM cliché highlighter is up to 38 patterns now tools.simonwillison.net/llm-…
178
166
37
2,648
161,412
Josh Robb retweeted
Going to bed late is associated with a higher IQ. Researchers at the London School of Economics analyzed thousands of individuals to map the relationship between circadian rhythms and cognitive ability. People with higher IQs are significantly more likely to be night owls. The data breaks it down by sleep schedules: • Very Dull (IQ < 75): Sleep by 11:41 PM • Normal (IQ 90–110): Sleep by 12:10 AM • Very Bright (IQ > 125): Sleep by 1:44 AM (and sleep in past 11:00 AM on weekends) Why? Evolutionary psychology. For 99% of human history, night was for sleeping. Artificial light didn't exist. Staying up late chasing complex thoughts, building projects, or solving problems is an "evolutionarily novel preference." People with higher general intelligence are more equipped to override ancestral instincts, break away from the traditional sun-up-sun-down routine, and adapt to a modern, 24/7 world. The early bird might get the worm. But the night owl gets the higher IQ score.
730
1,531
752
13,020
1,901,224
Josh Robb retweeted
“A developer with taste and discipline can use AI to produce more good work. A developer without either can produce a much larger disaster. AI can give a brilliant person the power of a team. It can also give a fool the output of a factory.”
1
2
13
899
Josh Robb retweeted
Having kids is crazy bc every once in a while they sleep like shit for a few days and then they immediately have a wild cognitive leap in skills
101
347
50
28,362
1,350,843
Josh Robb retweeted
I added a tiny "papercuts" cli tool that agents can use to complain about bullshit they encountered during work, like dead-end tool calls, broken links, or other frustrations. The models would usually just push through without mentioning any problems
95
116
43
3,681
388,879
Josh Robb retweeted
ok fine, i built it paste any recipe link, get the table, skip the life story
i can’t believe someone solved recipe design decades ago and everyone just decided to ignore it
201
860
101
11,613
524,955
For all the OpenLiveWriter fans out there on MacOS, i've got something cool coming your way ❤️ cc @shanselman
1
3
1
10
11,622
the era of autistic deep work is over your agents are more autistic than you and better at deep work welcome to the shallow work future total ADHD dominance
247
796
126
12,656
416,897
Josh Robb retweeted
On Still Burning I told Keith Adams this shift erases two of my books. Crafting code so it communicates to other humans was a real craft. But it's a bit like horseshoeing now. No leverage in being the best horseshoer, even the world's best.
6
6
1
54
10,962
“oh, every morning at 1:00am our language model regenerates the whole codebase from scratch based on the current requirements document. it’s more reliable than trying to make incremental edits”
16
52
28
713