The missing yardstick: why AI's benefits may stay concentrated
Most discussion of AI adoption focuses on the models: which is smartest, cheapest or most open. The quieter bottleneck is that there is no common yardstick for deciding which model is fit for which job.
Two things are missing.
A shared definition of the task. Work breaks into thousands of tasks, and every evaluator names and groups them differently. One calls it "claims triage", another "document classification", a third splits it into five sub-steps. Results measured against different definitions can't be compared.
A shared way of judging performance. Even for the same task, evaluators test differently and report differently. The bar also changes by industry: "summarise this document" means one thing for a marketing team and something much stricter for a lawyer or a bank's risk desk. A model that tops a general leaderboard may fail the version of the task that matters to you.
An example. Take a mid-sized insurer that wants AI to speed up claims processing. That work breaks into dozens of distinct tasks: reading medical reports, checking policy wording, flagging fraud, drafting letters to customers.
For each task it needs to know which model is good enough, which is cheapest, and which tasks should stay with people. It finds vendor claims it can't verify, leaderboards built on tasks that don't match its own, and evaluators whose categories don't line up. It has no in-house team to test models against its own work, and certainly not every quarter as new models arrive.
So it does one of three things:
Overpays: uses the most expensive model everywhere, just to be safe.
Stalls: the project never gets past a pilot.
Trusts a vendor: hands the decision to whoever sells it the product.
Why this matters for economic dispersion. Large technology firms and AI-native companies can build their own testing and work around the gap. Most of the economy cannot: mid-sized firms, regulated industries, and companies outside the tech sector.
If only well-resourced firms can tell which model fits which task, the productivity gains from AI will concentrate in them, even as the models themselves get cheaper and more widely available. Cheap intelligence spreads value only if ordinary organisations can confidently match it to their work.
The open question: who should define the tasks, set the evaluation standard, and vouch for the results, so that any organisation, not just the largest, can make that match with confidence?
Munish Khurana retweeted
New essay! A big-picture, speculative article on how most of the huge questions about the coming AI revolution aren’t really about AI in a narrow, technical sense - and why it will be a golden age for polymaths, philosophers, and moderate epistemic despair. Link: conspicuouscognition.com/p/m…
Leopold Aschenbrenner’s AI-focused hedge fund Situational Awareness suffered heavy losses in July 2026 AI stock declines, faced margin calls due to leverage, and sold the bulk (or essentially all) of its public equity portfolio to Ken Griffin’s Citadel; it retained private holdings, notably a large Anthropic stake, with assets dropping sharply. wsj.com
Aschenbrenner (then in his mid-20s), a former OpenAI researcher who left in 2024, published a widely read ~165-page essay titled “Situational Awareness” arguing for rapid AI progress requiring massive buildouts of chips, memory, power, and infrastructure. He launched the hedge fund of the same name around then (seeded with roughly $225 million from investors including Stripe’s Patrick and John Collison, Nat Friedman, and Daniel Gross). It grew extremely quickly—reports put peak assets as high as ~$45 billion around early July 2026—and delivered strong performance, including ~439% net returns after fees through the end of June 2026. cnbc.com
The portfolio reflected the essay’s thesis: concentrated, leveraged longs in the “physical layer” of AI (examples frequently cited include SK Hynix, Micron, Sandisk/Nebius, CoreWeave, and related names) paired with shorts on software companies expected to face commoditization (e.g., Adobe). Leverage amplified returns on the way up. techcrunch.com
In July 2026 a sharp sell-off hit AI-related equities (especially infrastructure and memory names). Key holdings fell sharply (many >30–50% in the period), while the software shorts moved against the fund. Leverage turned the drawdown into margin pressure from prime brokers (reports name Goldman Sachs, JPMorgan, Bank of America, and others). On or around July 24, Aschenbrenner sent investors a letter framing the sell-off as one of the better buying windows since early 2025 and inviting fresh capital. Days later the public book was sold. reuters.com
Citadel acquired the bulk of the public stock portfolio (estimates around a $16 billion public book before the trade) in what multiple outlets describe as a large block/forced or pressured sale facilitated by the brokers. Other parties (including Millennium and Jane Street in some accounts) were reportedly involved in discussions. After the transaction, Situational Awareness’s overall assets were reported around $10 billion. It kept its private investments, most notably a stake in Anthropic valued in the range of ~$5 billion in contemporaneous reports; the fund is described as continuing in a reduced form, more concentrated on private holdings. bloomberg.com
Munish Khurana retweeted
I'm actually fairly bearish on frontier lab valuations. I've never seen the reasons articulated to my satisfaction, so before I go to sleep, I wanted to quickly jot down my thinking here.
The basic issue is that the labs are highly unprofitable. This may seem like a simple point, but private market valuations can be relatively irrational; however, like with $SPCX, post-IPO pricing will likely be much more punishing, especially as the standard 6-month lockup period expires and selling pressure intensifies.
Many people claim that the labs have high margins. Yet even with high margins, a valuation of $1T would be justified only if the labs were doing nothing aside from serving inference (thus reducing costs only to those relevant to inference) and posting annual revenue numbers in the $100-200 billion range assuming ~80% gross margin and a 20x earnings multiple.
This assumption is obviously not true, because the frontier labs have to continually spend money training the next generation of models. This is because of market competition from runner-up firms. For example, if OpenAI had paused model development last year, there would no longer be any point in paying GPT-5 API prices when you can just use Qwen or Kimi instead for much cheaper. Thus, the labs are forced to invest ever-increasing amounts of money in model training, in a way such that at any given point of time, the amount you're forced to invest in the next model is dramatically higher than the amount of money you're actually making, because even if your revenue goes up with higher model capabilities, so do your future training costs. This is a profoundly punishing dynamic which severely penalizes frontrunners.
(There is also a related subpoint where frontier labs claim they can distill their leading models to win out at lower intelligence levels as well. This makes no sense because the revenue numbers involved are far too low when taking into consideration the rather low margin of such inference.)
Frontier lab valuations appear largely to be based on the assumption that as you scale up, the capabilities which emerge will be sufficiently general and profound that we'll see explosive growth (epoch.ai/publications/explos…) from things akin to AI agents starting and autonomously managing entire companies of subagents. But it's not clear to me that this is the case; indeed, as I mentioned in my previous post (xitter.cf/andrewho03/status/2082…), I believe that capabilities growth will be slower, spikier, and more data-limited than people currently assume. It may be the case that eventually we will see explosive growth of this nature with full automation of the economy, but at the very least my viewpoint implies much longer (multi-decade) timelines until we reach this point. It is not clear to me that the frontier labs will be able to operate unprofitably for so long, although I suppose maybe this foreshadows some sort of inevitable nationalization.
I also want to make a broader point about technological diffusion. The reason why technological diffusion is slow isn't just because, e.g., old people take a long time to learn how to use technology (although this is of course a contributing factor to some degree). In my view, it's because when a new, revolutionary technology comes along, the ways to incorporate that technology into subsequent developments are not always obvious, and in fact they cannot necessarily be arrived at through the application of pure reason. If they could be, then perhaps frontier models, at a certain point, would have a perfect understanding of how the LLM application layer should be developed, and they would then autonomously code, deploy, and sell such a layer.
But it seems more plausible to me that this diffusion is limited moreso by the hard problem of economic calculation--that is to say, the Hayekian notion through which the price system gradually promotes efficient allocation of resources and which cannot be simulated through central planning--and that even if we froze current capability levels at today's levels, it would take well over two decades to fully integrate in LLMs into our lives. Such a view is consequently rather bearish for the continued profitability of labs as it reduces their prospects for finding, say, something else comparable in profitability to coding agents, which seems to have been a somewhat lucky discovery by Anthropic to begin with. That is to say, even if you spam FDEs you aren't necessarily going to be able to just figure out the "correct" product shapes fast enough.
Overall, I don't think that people have clearly reasoned through their mental models for why lab equity should be worth as much as it currently is, and that if you actually bother to write down such a model, you may not arrive at the conclusion that you want to arrive at. This isn't to say that I don't expect AI to experience a huge (industry-wide) boom in the coming decades, but just that I'm not entirely sure I would buy OpenAI or Anthropic stock at latest valuations if I were given the opportunity to do so.
Of course, as an ex-lab employee, arguably this is talking against my own book; I should really be giving people more reasons to be bullish. But in the end, my influence is so small that it doesn't make a difference, so why not have some fun?
New blog post on what would be true about the world if trendline continues and leading lab hits $1T in revenue by the end of next year.
In other words, why compute might get 10x+ more expensive in coming years
dwarkesh.com/p/why-compute-m…
Munish Khurana retweeted
In just 9 years since the 2017 Transformer paper, we’ve encoded a significant portion of human knowledge: text, design, and reasoning into floating-point vectors in high-dimensional space.
LLMs don’t “know” things in the human sense. They navigate geometry: shifting attention, activating weight pathways, and recomposing representations via linear algebra and probabilistic sampling.
So let’s stop debating benchmarks and ask a harder question:
Freeze the world at 1905.
No relativity. No quantum mechanics. No modern physics.
Can a transformer, trained only on pre-1905 data, derive Einstein-level insights? (@demishassabis's idea that i love!)
Repeat this across key inflection points in science and engineering.
If it can, we’re looking at systems capable of genuine discovery under constraint, not just interpolation over compressed history.
That’s the real AGI test. @elonmusk @sundarpichai @satyanadella @ylecun
Munish Khurana retweeted
Tricking the US Government to protect frontier labs’ business model by using a China boogeyman is a mistake.
It is protecting the equity of 5,000 people who are investors in OAI and Ant at the sale of everyone else. This would be a terribly stupid decision.
Let the market sort this out!
The events of the past few weeks may simply mean that the frontier labs’ business model is not good and that their revenues aren’t sustainable.
That’s ok. It’s ok to have a business model that worked for some time then all of a sudden didn’t (remember Groupon)!
This still leaves a lot of value for a lot of other American companies:
- CPUs
- GPUs
- hyperscalers
- neoclouds
- rack manufacturers
- electricity and power
- skilled trades
- application layer AI
None of this value goes away. In fact, if the model layers’ costs decrease by 10-100x, competition would accelerate and I would argue that the value to everyone else will go up more than what is lost.
Let the free market sort this out vs allowing two companies whose stock is owned by less than 5,000 people decide this for America.








