Nice talk on the future of AI. Hinting that verticals will matter, it will not cure cancer soon. and we need new research into entirely new kinds of reasoning models.
I love the field! Room for innovation.
Alphabet's Chairman: LLM’s Aren't the Endgame youtu.be/E0ZdVnnMOw4
(PS. He made a comment that if capture a few % of productivity - it will pay off the investment so far. I am not sure of that, but I am moving away from what one could call the "politics" of AI - as it was affecting motivation).
Andrew retweeted
It's not an LLM.
As soon as inference involves a *search/optimization* creative solutions can be found.
AlphaZero performs search.
Regular LLMs don't.
Andrew retweeted
Software engineering is dead... In other news, a software engineer with 18 years of experience (me) just spent 15 hours refining a big new feature, asking questions that the frontier models hadn't even considered. Software engineering isn't going anywhere.
Andrew retweeted
To me, even “artificial intelligence” was a stretch given LLMs are probability distributions and do not meet our common understanding of “intelligence.” Hallucination is a feature not a bug; generating the next token and the next also.
Renaming LLMs to “superintelligence” 😱
Andrew retweeted
That trillion dollar IPO is looking less likely by the day...
If Thomson Reuters can do this, any large company can, which means the enterprise model for all the big AI cos is largely a bust.
That's all their cash flow gone, so good luck paying for all those datacentres... 😉
🦔Thomson Reuters just built its own AI model for about $40 million over two years. The final training run cost $450,000. They started with Qwen, an open-source model from China's Alibaba, and trained it on their own legal and news content from Westlaw, Practical Law, and Reuters. The company said the move is about reducing its dependence on Anthropic. Their CTO compared paying for outside AI to being a permanent tenant versus owning the building. Enterprise customers broadly have been cutting spending on OpenAI and Anthropic and switching to cheaper alternatives.
My Take
I covered the DeepSeek pricing collapse a few months ago and said the frontier labs would have a hard time defending premium API pricing once open-source got good enough. Thomson Reuters just did exactly what I expected someone to do. They grabbed a free model, trained it on their own stuff, and now they're pulling back from Anthropic. $40 million, done. Anthropic charges that in API fees from a handful of big customers in a year.
The $190 to $200 billion revenue projection Anthropic is selling to IPO investors assumes companies like Thomson Reuters keep paying. They just stopped. And Thomson Reuters put their model on Hugging Face for academics to use, which means the playbook is now public. I think the frontier API business has maybe two or three years before most large companies with good data figure out they can do this themselves, and the ones who move first are going to pressure the ones still paying full price to ask why.
Hedgie🤗
Andrew retweeted
The best founders I’ve met all have a very low need for social approval, and this is incorrectly labelled as a red flag by the industry.
Most smart+ambitious people are terrified of looking like a fool or bothering people, but many of the great CEOs we’ve seen would send 10 follow-up emails to a dream hire or pitch their product cold to a prospective customer in an elevator without any hesitation; this behavioural quirk allows them to bypass the politeness instinct that slows down by-the-book entrepreneurs.
When you’re assessing the strength of a founder, try to look for signs of shameless persistence. This ability to shrug off the fear of being annoying will speed up your progress like nothing else.
Andrew retweeted
If you're 17 (or any age) and you want to learn to build LLMs from scratch, read chapters 15-16 of Deep Learning with Python, available online here: deeplearningwithpython.io/ch…
In particular, chapter 15 has one of the best explanations of WHY dot-product attention works that you'll find anywhere.
chatgpt.com/share/6a8d227b-4…
Caught GPT out here. It cannot see the obvious. That if the account was deleted it would no longer be possible to see that user's posts and count the likes.
Andrew retweeted
This is not an approach that will only shine on ARC 3. This is where most of AI is headed in the long run.
xitter.cf/fchollet/status/207277…
Andrew retweeted
Our general approach is to automate the experimental loop. We think this approach is broadly applicable across many different fields of science and engineering. We’ll initially focus on ML research and engineering, but believe the approach can help with important subproblems in nearly every one of the fourteen NAE Grand Challenge problems. We think doing this well requires strong expertise in machine learning as well as large-scale systems.
Andrew retweeted
I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a "harness"), running at inference time, orchestrating thousands of calls to a neural network for any given task, is the exact definition of a "neurosymbolic architecture"
Andrew retweeted
You have Dwarkesh predicting Anthropic is going to make well over 100B in revenue this year.
and then you have this chart.
Andrew retweeted
Yet another paper argues that LLMs aren’t close to doing real discovery.
MIT and Harvard argue LLMs are nowhere near doing real scientific discovery.
They published a paper called “Evaluating Large Language Models in Scientific Discovery.”
Every week, tech labs claim an LLM has made a breakthrough in biology, physics, or chemistry.
But this proves they are faking it.
For years, AI benchmarks have tested models using static, multiple-choice science trivia. Models ace these tests, leading everyone to believe AI is right on the verge of autonomous scientific discovery.
Researchers built a new evaluation framework called SDE to test what happens when you take LLMs out of the multiple-choice quiz and put them into real, open-ended research projects.
They tested frontier models across biology, chemistry, materials science, and physics.
The results are sobering.
When forced to handle the actual loop of discovery—proposing a testable hypothesis, designing simulations, running experiments, and interpreting ambiguous results iteratively, current LLMs fall apart.
There is a massive, glaring performance gap between passing standard science benchmarks and doing real science.
Why do they fail? Because real science requires iterative reasoning, handling imperfect evidence, and adapting to unexpected observations.
LLMs are built to predict the next token based on existing internet data. They can regurgitate a textbook explanation of photosynthesis or quantum mechanics instantly.
But when placed inside an uncharted loop where the textbook doesn't have the answer yet, they hit a wall.
Worse still, the researchers discovered diminishing returns. Simply scaling up model sizes and adding raw compute isn't fixing the gap. Top-tier models from different providers share the exact same blind spots.
We are miles away from general scientific superintelligence.
The tech industry is selling a narrative that AI is about to automate labs, run clinical trials, and invent materials on autopilot.
But right now, AI isn't doing science.
It's just remembering it.
Andrew retweeted
Claiming that human code is full of errors, agentic code is error free all to pitch your startup that positions itself to rewrite code w AI is bold.
Because agentic code is as buggy as human code trained on, in fact introduces new, harder to spot bugs errors and vulnerabilities.
As of today, let’s assume 99% of all code written on the internet is still human written and 1% is agentic.
That means 99% of code is sloppy and error ridden. Humans make errors that agents don’t.
As agentic code replaces human code, both the error rates and security incidences will GO DOWN.
There will be new forms of attack vectors by swarms of agents but I would not shut down the internet because of that because companies will have sophisticated swarms of agents to protect them.
If everyone uses commercially available open or closed source models, a stalemate is the most likely terminal outcome.
That said, the immediate priority is obvious. We need to rewrite all the code in all the enterprises. Period. Non negotiable.
So let’s get on with it, close the security holes and flip to a 100% agentic code universe which, still may not make useful or tasteful products, but at least won’t have the errors that humans leave behind as we work.
Andrew retweeted
People keep on telling me that my message about AI is undercutting my own books. Those people do not understand how agents work and who actually controls them. You can't tell an agent to be clean. You have to measure the cleanliness that they produce and have them correct failures of cleanliness.
Without such constraints agents are more than happy to build big balls of mud that they can't maintain.
They are certainly not saying "never". They are however responding to all the many comments from current LLM makers and AI enthusiasts that these jobs will be replaced in a few months or years.
