
AI Engineer. Helping you demystify LLMs and AI. Taught hundreds to code @FlatironSchool Co-Founder @Crypto_NYC
NYC
Joined July 2007
- Tweets5.3K
- Following828
- Followers923
- Likes2.8K
Blake Johnson retweeted
My reflections on OpenAI Chief Scientist Jakub Pachocki’s “An Alien Mind” on AI alignment, the pace of progress, and what this moment asks of us as humans. We’ll need breakthroughs in how we cooperate and adapt, alongside breakthroughs in AI.
Blake Johnson retweeted
I burned $13,836 worth of Claude Code tokens in June.
The same work on open models: $3,106.
That's $10,730 left on the table — one month, one laptop.
So I built the instrument that shows you the gap. Open source, 100% local. Launching today 🧵
Tally is launching an ICO.
We believe we've built the best on-chain mechanism for raising capital via tokens — and we're going to prove it by using it ourselves.
Over the next 60 days, we're documenting every step of what it looks like to launch a token the right way, in the United States, in 2026.
We're demystifying a process that has historically be rife with abuse by bringing all the steps out into the open.
Episode 1 👇
Blake Johnson retweeted
At @Clay we’re always thinking about growth – and I’m particularly proud today that we can help our co-workers grow their wealth. Today, we’re announcing our second employee tender offer at a $5B valuation.
That’s more than 3x our last tender valuation, in just 9 months!
We’re able to do this because we have best-in-class growth while being extremely efficient.
We’ve crossed a big milestone – 100M in ARR with an Enterprise NRR of more than 200% and no churned enterprise customers – and we want to give the team that got us here the ability to realize gains from some of their vested equity.
Some companies worry that offering employees early liquidity leads to churn or weaker motivation, but for us it’s been the opposite. In fact, the last tender was massively under-subscribed.
When people feel trusted, supported, and happy, they stay longer and build with more conviction – just check out our Glassdoor reviews (p.s. we’re hiring!)
The tender will allow employees to sell up to $55M of Clay shares at a $5B valuation, and is led by DST Global, with participation from @conviction, @AvraCap, @OpCo_VC and @Frontlinevc, alongside a stellar roster of angels and customers, including Stripe's @chughesjohnson, Figma CMO @SheilaVashee, Superhuman CEO @shishirmehrotra, and product leader @lennysan.
Thank you to our team, community, and customers for making this milestone possible. We're hyped for the journey ahead!
→ Check out The New York Times feature by @m_delamerced: nytimes.com/2026/01/28/busin…
Absolute banger of a post by one of my favorite accounts @divya_venn
It’s a new spin on an old tale, the frog and the scorpion.
Why can’t we help ourselves?
2
5
639
interesting post on EGI, a much more hand wavy version of AI 2027 but good food for thought
I spent the past year building AI for robots at Tesla Optimus and Dyna
Now I'm introducing ANDROID DREAMS: an essay of my predictions for the next 20 years of robotics, inspired by Situational Awareness and AI2027.
I predict EGI by 2031 and more robots than humans by 2045👇🏼
1
234
Fascinating description of the inherent computational tradeoffs in the human memory system and how it relates to designing memory for AI
"Long term memories have to be able to store the traces or paths to distributed neural representations over long periods of time. However, the entire function of continual learning is to keep updating and relearning these precise representations, thus slowly causing the stored memories or paths to the memories to drift out of date. This means there is a deep trade-off – the more continual learning that you do, the harder it is to maintain consistency and availability of old memories. It is possible to try to design some kind of synchronization system that regularly updates all memories to keep them in sync with the drifting representations, however this eliminates the main computational advantages of long term memory storage – that you can largely leave it alone unless needed. Having to constantly refresh all of your memories imposes an increasingly large burden as the memory size gets bigger."
1
67
Some fun ideas in this post henrikkarlsson.xyz/p/wordles…
"A more technical way of saying this is that our (non-verbal) thoughts seem to behave as vectors; when a cluster of neurons fire together, that pattern is like an address pointing toward a point in a high dimensional space. But when we convert our thoughts to words, we convert that vector into a scalar.
The discussion of vectors and dimension reduction also has an interesting parallel to an ongoing discussion in AI research. When a large language model calculates what to output, the “thinking” happens in high dimensional space where vectors are passed from layer to layer. At the final layer, that high dimensional representation is collapsed into a token—the written output. When this happens, enormous amounts of information is lost: the residual stream contains over a thousand times more information than gets encoded into the token! That lends some support to the idea that non-verbal (partly unconscious) thinking might be more information rich in humans, too."
1
1
95
Omg somehow I missed the obvious that alpha schoolers don’t have homework! I can’t even imagine how much joy this would have brought me. I am going to shill alpha 10X harder. Free the prisoners
Pretty convenient for me that I didn’t believe in homework before AI made cheating on it trivial.
My argument against homework is “6.5 hours is a long time to spend in school. That should be more than sufficient for class and study time for a K12 general education.”
1
2
218
Blake Johnson retweeted
This is the most valuable resource I have ever created
I wrote a brand new, extremely detailed Notion guide on every single strategy to save money on taxes
The best of my content in a single place
Want a free copy?
- Like / RT this post
- Reply with "GUIDE" and I'll DM you
Attention is all you need! What an absolute banger of a title to kick off the AI era. Did they know?
love is attention:
the transformer taught us something profound: attention is all you need. but maybe we knew this all along, just in a different language.
when an LLM processes text, it doesn't read sequentially like we intuitively think. it attends – it looks at every token and asks "what matters here to understanding this?" it builds a web of relevance, weighing what connects to what. some words light up for others. some fade into background noise. the model learns what to notice.
this is love.
love isn't about being in the same room or even the same timezone. it's about what you attend to. when you love someone, they're not just in your field of vision – they're in your attention matrix.
the opposite of love isn't hate – it's inattention. it's when someone or something becomes background tokens, processed but not weighted, present but not salient.
LLMs have context windows, and so do we. we can only hold so much in our attention at once. this is why love is work. this is why it matters who and what we choose to attend to. every moment of focus is a moment of resource allocation. every conversation is a forward pass through the network of what we care about.
and like transformers, we learn through attention. we become what we attend to. the model updates its weights based on what patterns matter. we update ourselves based on what we notice, what we care about, what we choose to see.
when you pay attention to someone – really attend, not just wait for your turn to talk – you're running a query against everything you know about them, retrieving relevant context, weighing what matters now. you're computing understanding in real-time.
maybe that's why "paying attention" is the right phrase. it costs something. it's a transaction where you give your scarcest resource: the focus of your consciousness in this moment.
the AI researchers were right. attention is all you need. for intelligence. for understanding. for love.
the architecture of care is the same as the architecture of comprehension: notice what matters, weigh it appropriately, let it transform you.
so when someone gives you their full attention – when they close the laptop, silence the phone, make eye contact, and really listen – they're doing something technically miraculous and deeply ancient. they're running you through their attention mechanism. they're saying: right now, in this moment, you are the highest-weighted token in my context window.
and that's love.
1
1
105
Replying to @karpathy
Something I am experimenting with. I copy pasted:
1) the full podcast transcript
2) the bitter lesson blog post
3) my full post above
To ChatGPT. The interesting part is you can fork the conversation context to ask any questions and take it in whatever direction with chat:
chatgpt.com/share/68dd6833-6…
2
1
161
Fascinating discussion of the Dwarkesh/Sutton interview. Karpathy’s point about animals not being booted up with a blank slate but instead millions of years of evolution which is comparable to our use of pre-training seems apt.
Finally had a chance to listen through this pod with Sutton, which was interesting and amusing.
As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed, just look at LLM scaling laws where if you put compute on the x-axis, number go up and to the right. So it's amusing to see that Sutton, the author of the post, is not so sure that LLMs are "bitter lesson pilled" at all. They are trained on giant datasets of fundamentally human data, which is both 1) human generated and 2) finite. What do you do when you run out? How do you prevent a human bias? So there you have it, bitter lesson pilled LLM researchers taken down by the author of the bitter lesson - rough!
In some sense, Dwarkesh (who represents the LLM researchers viewpoint in the pod) and Sutton are slightly speaking past each other because Sutton has a very different architecture in mind and LLMs break a lot of its principles. He calls himself a "classicist" and evokes the original concept of Alan Turing of building a "child machine" - a system capable of learning through experience by dynamically interacting with the world. There's no giant pretraining stage of imitating internet webpages. There's also no supervised finetuning, which he points out is absent in the animal kingdom (it's a subtle point but Sutton is right in the strong sense: animals may of course observe demonstrations, but their actions are not directly forced/"teleoperated" by other animals). Another important note he makes is that even if you just treat pretraining as an initialization of a prior before you finetune with reinforcement learning, Sutton sees the approach as tainted with human bias and fundamentally off course, a bit like when AlphaZero (which has never seen human games of Go) beats AlphaGo (which initializes from them). In Sutton's world view, all there is is an interaction with a world via reinforcement learning, where the reward functions are partially environment specific, but also intrinsically motivated, e.g. "fun", "curiosity", and related to the quality of the prediction in your world model. And the agent is always learning at test time by default, it's not trained once and then deployed thereafter. Overall, Sutton is a lot more interested in what we have common with the animal kingdom instead of what differentiates us. "If we understood a squirrel, we'd be almost done".
As for my take...
First, I should say that I think Sutton was a great guest for the pod and I like that the AI field maintains entropy of thought and that not everyone is exploiting the next local iteration LLMs. AI has gone through too many discrete transitions of the dominant approach to lose that. And I also think that his criticism of LLMs as not bitter lesson pilled is not inadequate. Frontier LLMs are now highly complex artifacts with a lot of humanness involved at all the stages - the foundation (the pretraining data) is all human text, the finetuning data is human and curated, the reinforcement learning environment mixture is tuned by human engineers. We do not in fact have an actual, single, clean, actually bitter lesson pilled, "turn the crank" algorithm that you could unleash upon the world and see it learn automatically from experience alone.
Does such an algorithm even exist? Finding it would of course be a huge AI breakthrough. Two "example proofs" are commonly offered to argue that such a thing is possible. The first example is the success of AlphaZero learning to play Go completely from scratch with no human supervision whatsoever. But the game of Go is clearly such a simple, closed, environment that it's difficult to see the analogous formulation in the messiness of reality. I love Go, but algorithmically and categorically, it is essentially a harder version of tic tac toe. The second example is that of animals, like squirrels. And here, personally, I am also quite hesitant whether it's appropriate because animals arise by a very different computational process and via different constraints than what we have practically available to us in the industry. Animal brains are nowhere near the blank slate they appear to be at birth. First, a lot of what is commonly attributed to "learning" is imo a lot more "maturation". And second, even that which clearly is "learning" and not maturation is a lot more "finetuning" on top of something clearly powerful and preexisting. Example. A baby zebra is born and within a few dozen minutes it can run around the savannah and follow its mother. This is a highly complex sensory-motor task and there is no way in my mind that this is achieved from scratch, tabula rasa. The brains of animals and the billions of parameters within have a powerful initialization encoded in the ATCGs of their DNA, trained via the "outer loop" optimization in the course of evolution. If the baby zebra spasmed its muscles around at random as a reinforcement learning policy would have you do at initialization, it wouldn't get very far at all. Similarly, our AIs now also have neural networks with billions of parameters. These parameters need their own rich, high information density supervision signal. We are not going to re-run evolution. But we do have mountains of internet documents. Yes it is basically supervised learning that is ~absent in the animal kingdom. But it is a way to practically gather enough soft constraints over billions of parameters, to try to get to a point where you're not starting from scratch. TLDR: Pretraining is our crappy evolution. It is one candidate solution to the cold start problem, to be followed later by finetuning on tasks that look more correct, e.g. within the reinforcement learning framework, as state of the art frontier LLM labs now do pervasively.
I still think it is worth to be inspired by animals. I think there are multiple powerful ideas that LLM agents are algorithmically missing that can still be adapted from animal intelligence. And I still think the bitter lesson is correct, but I see it more as something platonic to pursue, not necessarily to reach, in our real world and practically speaking. And I say both of these with double digit percent uncertainty and cheer the work of those who disagree, especially those a lot more ambitious bitter lesson wise.
So that brings us to where we are. Stated plainly, today's frontier LLM research is not about building animals. It is about summoning ghosts. You can think of ghosts as a fundamentally different kind of point in the space of possible intelligences. They are muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity's documents with some sprinkle on top. They are not platonically bitter lesson pilled, but they are perhaps "practically" bitter lesson pilled, at least compared to a lot of what came before. It seems possibly to me that over time, we can further finetune our ghosts more and more in the direction of animals; That it's not so much a fundamental incompatibility but a matter of initialization in the intelligence space. But it's also quite possible that they diverge even further and end up permanently different, un-animal-like, but still incredibly helpful and properly world-altering. It's possible that ghosts:animals :: planes:birds.
Anyway, in summary, overall and actionably, I think this pod is solid "real talk" from Sutton to the frontier LLM researchers, who might be gear shifted a little too much in the exploit mode. Probably we are still not sufficiently bitter lesson pilled and there is a very good chance of more powerful ideas and paradigms, other than exhaustive benchbuilding and benchmaxxing. And animals might be a good source of inspiration. Intrinsic motivation, fun, curiosity, empowerment, multi-agent self-play, culture. Use your imagination.
126
Just read a fascinating post that made me do a 180 on how AI adoption will affect the labor market over the next few years.
Radiology looked maximally automatable—CheXNet-style breakthroughs and 700+ FDA-cleared imaging AIs, including a few autonomous systems. Yet demand for human radiologists is up: 2025 set a record 1,208 U.S. residency positions, vacancy rates remain high, and average pay sits around $520k. The long-predicted “replacement” moment didn’t arrive.
The most underappreciated blocker is insurance. Many malpractice policies and payer rules exclude AI-only reads, insisting on a physician of record. Even where autonomous software is technically capable, institutions keep humans in the loop because liability and reimbursement hinge on them. The FDA holds autonomous systems to a higher bar, retraining can trigger fresh approvals. Adoption mirrors this reality: only ~48% of radiologists use any AI at all, and only ~19% of pilots report “high” success.
Benchmarks haven’t translated cleanly to bedside. Today’s tools are islands of automation—mostly single-task—with coverage clustering on stroke, breast, and lung while many subspecialties are sparse. Generalization is shaky: 38% of models were validated at a single site, and performance can drop ~20 percentage points out of sample; datasets under-represent kids, women, and minorities.
What AI is changing is the task mix, not headcount. Only ~36% of a radiologist’s day is pure image interpretation; the rest is protocols, communication, teaching, and procedures. When hospitals went filmless, productivity rose 27–98% with no layoffs, and imaging per thousand insured patients climbed ~60% in the 2000s. Cheaper and faster tends to mean more imaging—so, near term, AI plus radiologists beats AI versus radiologists.
2
1
63













