CTO & cofondateur de https://xitter.cf/t.co/fBdEU8XbjQ & https://xitter.cf/t.co/ohuKRlYnYm
Aix en Provence
Joined August 2021
- Tweets1.5K
- Following109
- Followers113
- Likes614
Thomas Mahier retweeted
There’s an ocean between what models can do if they have to a ton of work but have a reference implementation, and when you want to design/make something new.
Thomas Mahier retweeted
Four Gemini 4 Argon agents, one shared Minecraft world, one goal: defeat the Ender Dragon. 27 hours and 2,783 self-written programs later, the dragon fell.
Thomas Mahier retweeted
It’s actually so hard to efficiently utilize 8 sub agents.
All this talk of swarms and you really need to engineer how work gets splits up, is synced up and balances thinking vs compute bottlenecks.
Efficiency is hard!
Thomas Mahier retweeted
i don’t understand why people say we’ve used up the internet for pretraining. we’re making exponentially more internet every day now
Thomas Mahier retweeted
If you haven't accepted that these things are smarter than you, even in your own domain of expertise, I'm sorry. And if you have, but still believe you should be making all the meaty calls in that domain, I'm sorry too. Both are hard pills to swallow.
Many saw this coming earlier, but the turning point for me was the Fable/Astra generation. Until then, I was in a comfortable sweet spot: an expert who still knew better, but who could leverage the amazing acceleration agents brought.
After that, well, once reality sank in, embracing it wasn't that hard. Suddenly, you're in a whole new greenfield where anything feels possible, and your experience gives you a head start.
I think this click moment will come for everyone. Some have already had it, and many others haven't yet. But reality catches up, sooner or later.
And remember, Fable/Astra will be rusty, outdated models tomorrow. Keep that in mind when placing your bets.
Thomas Mahier retweeted
all technical work is becoming evals. your primary job as an engineer is now building loops that make coding agents effective maintainers of the systems you own. models are immensely capable - given clear instructions, the right environment, and a way to verify their work, agents can complete open-ended, long-running tasks. if you still think coding agents are shipping “slop”, it simply means one of these three things is missing.
if you build product, you might need to restructure your codebase and give your agents fast, high-fidelity ways to test changes. if you’re a researcher, you might spend most of your time building pipelines for data synthesis and environment construction - experiments are the bottleneck, not ideas.
the art is translating the real-world problem into the right eval: will the agent understand your exact criteria for when a task is sufficiently complete? and when the frontier shifts, do these criteria change?
every engineer is an AI engineer now. in the long run, the ones who build the machine that builds the machine will be most effective. of course, the best will adapt. but if you’re not actively thinking about how to make agents better at your work, or you’re relying on other people to do it for you, you’re probably ngmi.
Thomas Mahier retweeted
Why are you still using pull requests?
Is it for code review?
If so, do you really think your or your colleagues catch bugs and mistakes that an Opus 5.5 or Fable or Astra won't find? You can literally spawn all three of them and have them check four times each.
(Also, don't tell me you don't have a "stamp plz" Slack channel or group chat somewhere.)
Is it for knowledge sharing?
Surely there's better ways for that now than the PR, no? We have models that can generate explainer videos, presentations, podcasts, games, quizzes, walkthroughs, ... for any piece of code it can reach.
Is CI the reason?
Well ackshually, CI means continuous integration and its original meaning is the *opposite* of having long-lived branches and PRs.
But, leaving that aside, your agents are hopefully already running the tests before they push something, and hopefully they can run as long as they want and don't block you so they can run the whole test suite and generate "irrefutable proof" and end-to-end demo videos and so on... And so they run the tests and confirm and then you push to a branch and everything runs again and then you merge and then it runs again? That doesn't make sense, does it?
Maybe you work in open source and you accept contributions, in which case I can't speak to the value of pull requests. It's been 2 years since my last OSS contribution. But from what I heard, the time of the PR is coming to an end in OSS too?
I honestly don't get why people still stick to PRs.
With agents you can now move 10x faster, you can finally do real continuous integration, and get more confidence in the changes the agent made.
Don't slow yourself down by having all the changes pile up in a PR.
Thomas Mahier retweeted
Scott was 2 months early on OpenClaw, so you better sit up when he starts talking about Pi Durable
youtube.com/watch?v=BLRReTwV…
Thomas Mahier retweeted
For your weekend reading: Build with Pi Durable on Cloudflare, a walkthrough of how to create a multiplayer chat agent, including subagents, on Durable Objects.
earendil.com/posts/pi-durabl…
Thomas Mahier retweeted
Should Europe leave frontier AI to the Americans and the Chinese?
The idea has gained ground in Europe. It is a fundamental mistake.
Build or surrender: for now, the choice is still ours.
Full article below.
Thomas Mahier retweeted
If you're wondering what the frontier labs mean when they say "RSI" (recursive self improvement) is coming within the next 12 months, it's this:
- Labs define RSI as the models making models better
- 100% of experiment coding is now automated
- 95% of experiment execution is now automated
- 20% of experiment ideation is now automated
The last bullet is the current bottleneck. As we've seen with nearly all definable benchmarks -- if we can reach 20%, we can reach 100%.
The current way to measure experiment ideation is on fixed-point synthetic evals. "How many times does AI have to try before it can reproduce a known-thing?"
This is not a great way to capture the true essence of RSI which I believe is about open-endedness -- asking the right questions no one has ever asked. No one has formalized this, much less in eval form. (We are trying with ARC-AGI!)
The current paradigm of RSI is probably best understood as trying to find a recipe for self-improvement within a specific domain.
But assuming we find it in the next 12 months, without more breakthroughs on generalization or open-endedness, we (AI + humans) are going to need to keep collaborating to scale that recipe horizontally into other data domains.
Thomas Mahier retweeted
insane what can now be done by small, determined teams with great RL taste
trained with `verifiers` + `renderers` :)
We are releasing Ultramarine 1 Flash, a frontier-class model built for knowledge work. It is open-source and achieves similar performance to Fable 5, while being 60x cheaper.
It was post-trained by a team of 5 for ~$300k all-in. Available on API now, with weights releasing in 2 weeks.
Thomas Mahier retweeted
String theory is what happens when an intelligent system is starved of sensory input and starts hallucinating. Intelligence needs grounding.
For this reason, intelligence is practically never the bottleneck to solving impactful problems. The bottleneck is iterated contact with reality.
Thomas Mahier retweeted
DeepSeek V4.1 Flash from @deepseek_ai on ARC-AGI (Verified):
- ARC-AGI-3: 3.3%, $878 (provider adapter harness), 0.9%, $1.2k (standard harness)
DeepSeek V4.1 Flash is the first model we've evaluated on ARC-AGI-3 to score above 3% for under $1k in total evaluation cost.
Thomas Mahier retweeted
We in Europe have a lot of problems in creating companies. But a big one was getting funding to pay programmers and create big companies. I believe that with AI, if people change mentality a little bit, even just the young folks, we can move a lot of IT back to Europe.
Thomas Mahier retweeted
On the one hand, I think a strong case can be made that we have made much less progress on AI capabilities than many assume. Frontier AI systems still struggle to learn from limited examples, learn continually from experience, and interact effectively with the physical world. More broadly, their competence is highly uneven. Superhuman performance on tasks they excel at (e.g., maths and coding, where training data can be generated at scale) doesn’t mean they will excel at the much messier tasks involved in most white-collar work.
On the other hand, nobody should find this assessment that comforting. It suggests there is vast room to improve the capabilities of existing systems by changing their design and algorithms. In other words, we could have vastly more powerful systems with the same amount of computing power.
Given that many of the world’s smartest people, assisted by already very capable AI systems and backed by vast resources, are working to accelerate AI research and development, even those sceptical of the current paradigm need to be prepared for the possibility of explosive change once certain capabilities are cracked.
Thomas Mahier retweeted
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel.
- Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all.
- Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract.
- Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code.
- And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while.
The funnel is driven by a combination of factors:
1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel.
2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel.
3. Access. Free-tier, paid-tier, internal.
So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
Judging by my tl there is a growing gap in understanding of AI capability.
The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code.
But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along.
So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions.
TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.
Thomas Mahier retweeted
🚨 CHINA JUST OPEN SOURCED AN OCR THAT EATS 100 PAGE PDFS IN ONE PASS
It’s called Unlimited OCR. Only 3B params. Runs locally.
Every other OCR tool chops your doc into pages and loses the thread. This one reads the whole thing in one continuous pass.
→ One shot full context parsing (32K context window)
→ Multilingual out of the box
→ 93% on the standard parsing benchmark (6 points over baseline)
→ Under 0.11 error rate past 40 pages
→ Runs 100% locally on your own hardware
→ Works with Transformers, vLLM, SGLang, Docker, Ollama, llama.cpp
Traditional cloud OCR (Textract, Google Vision, Azure Doc Intelligence) costs $1.50 to $15 per 1,000 pages.
This runs on your machine. For free. Forever.
Baidu built it explicitly to push DeepSeek OCR one step further. Already at 1.9M downloads on Hugging Face and most people have no idea it exists yet.
100% open source.
