The AI deployment and education firm. 500k+ AI practitioners trained since 2019. We deploy AI inside enterprises, funds, and PE portfolios.
United States
Joined April 2019
- Tweets24.8K
- Following2.1K
- Followers41.1K
- Likes1.7K
we paid opus 5.5 $1 to write one youtube script and almost 90 cents of it went to work we never saw
that's the hidden work. before a model answers it thinks, plans and checks itself, and you pay for every token of that even though you only ever read the final script
on xhigh ef
Everyone said Anthropic nerfed Claude. Here's what actually happened.
Every few months after a new Claude model comes out, the same complaint shows up. "They nerfed it." This spring it hit Claude Code hard. It forgot things, repeated itself, picked weird tools, and lots of peopl
We read all 26 posts on Anthropic's engineering blog. One secret in there changed how we test our AI agents.
It starts with a simple question. Your agent solves a task 75% of the time. If you run it 3 times, what are the chances it works all 3 times? A) 75% B) 98% C) 42% D) 25%
Your AI code reviewer isn't finding new bugs. 59% of what it flags is old code.
Anthropic's Claude Code team had one AI agent fix a pull request while another AI agent reviewed it, round after round, until the code came back clean. It never got there. They went back and forth mo
Stanford tracked millions of paychecks. Young workers in AI-heavy jobs are 19% behind.
The part that stuck with us is who isn't behind. Experienced people in those exact same jobs show no gap at all. Same jobs, same AI, and only the 22 to 25 year olds lost ground.
The study poi
we had gpt and claude grade the same 6,600 scripts and each one picked its own family
the claude judge's top 5 were all opus 5.5. the gpt judge's top 4 were all gpt. same scripts, and neither judge knew which model wrote what
one model, gpt-6 sol, came 4th with the gpt judge an
Your CLAUDE.md is the reason Claude ignores your rules.
You write "NEVER touch the migrations folder." Claude touches the migrations folder. So you add it again in caps and it still happens. Anthropic's own Claude Code guide explains why, and it's not the model. Bl
we turned 33 AI models up to max effort and had them all write the same youtube scripts
you'd think the writing would get noticeably better. it mostly didn't
the big change was the word count. on low effort models often miss the word count you ask for. on max they land close. t
Anthropic's engineers don't prompt Claude Code like you do.
Most of us type "make the tests pass" or "make it look better" and then spend an hour cleaning up after it. Anthropic wrote guides on how their own teams use Claude Code, and the best part is easy to miss. They have the
the most surprising thing in our writing benchmark wasn't who came first
opus 5.5 came first. no shock there
it's that gpt-6 sol came 4th for 13 cents a script. opus costs $1
and on hooks, facts and AI slop they scored basically the same
opus only really pulled ahead on sound
𝗬𝗼𝘂 𝗱𝗼𝗻'𝘁 𝗻𝗲𝗲𝗱 𝘁𝗼 𝘁𝗿𝗮𝗶𝗻 𝗮 𝗺𝗼𝗱𝗲𝗹 𝘁𝗼 𝗯𝗲𝗰𝗼𝗺𝗲 𝗮𝗻 𝗔𝗜 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿.
Most people think the job is hard math and huge computers. So they spend months on math classes before they build anything. Then they get the job and find out the model is alrea
After four months and more than 500 hours of work, ToneBench is finally live.
ToneBench started as @Whats_AI initiative inside Towards AI. Writing is a big part of what we do: newsletters, books, courses, the articles we edit every week, and Louis's YouTube videos. We use AI to draft and edit much of it, and Louis got tired of guessing which model to use. So he set out to measure it.
Plenty of benchmarks tell you which model codes best. Very few measure writing, even though AI writing that sounds like AI is one of the most common complaints about these tools. Phrases like "In today's fast-paced world" are easy to spot. The bigger problem is writing that stays vague, repeats itself, makes things up and sounds like nobody in particular.
ToneBench measures all of that. Each model gets a brief and a research packet and writes a full YouTube script in the voice of our co-founder, Louis-François Bouchard. We ran 78 models and 6,350 drafts, and 4 judges from 4 different companies scored every draft blind.
Louis also presented it today on stage at AI Engineer NYC.
Three results from the current board:
Claude Opus 5.5 at xhigh matches max effort on Writing Elo and costs $1.00 a script instead of $2.58.
GLM-5.3 Flash ranks 12th of 78 models for about 2 cents a script.
GPT-6 Sol at medium uses the least stock AI phrasing of any model.
The benchmark covers one voice on one task, so treat it as a starting point when you pick a model for your own writing.
See the full leaderboard and compare any two models: benchmark.towardsai.com/
Louis's AIE NYC session: ai.engineer/nyc/2026/schedul…
The recording will be on YouTube soon.
Louis posts results for every new model he tests on X: xitter.cf/Whats_AI
#AIEngineerNT #AIEngineerNYC #AIENY
It's LIVE. I can finally stop saying "our internal writing benchmark".
It's called ToneBench, and it's public: benchmark.towardsai.com
We ( @towards_AI ) were SO excited to release it first at #AIEngineerNYC, during my talk (a few minutes ago!). Thank you @swyx and the whole team @aiDotEngineer for having me!
5 months of work to answer one question: can any model write like me?
78 models + every thinking level. 6,350 scripts, each scored blind by 4 judges. That's 25,400 judgments. It's A LOT.
A few things you'll find in there right now:
- Claude Opus 5.5 is #1, and xhigh ties max for $1.00 a script instead of $2.58.
- GPT-6 Sol is the best model outside Claude, at 13 cents a script.
- GLM-5.3 Flash is 12th for 2 cents a script. Yes, 2 cents.
- Be careful with GPT as a judge. It REALLY likes itself.
You can compare any two models, see what each thinking level buys you, and read the full methodology. We share everything on how we built it, our decisions, and the full setup.
It measures my voice on my scripts, so it's one writing data point, not the final truth on which model is better. But if you write with AI, it should help you find your model.
The talk will soon be on YouTube, hopefully @swyx ? ;)
I'll keep adding every new model as it comes out, and I'll post the results here first. :)
𝗩𝗶𝗯𝗲 𝗰𝗼𝗱𝗲𝗿𝘀 vs. 𝗣𝗿𝗼 𝗱𝗲𝘃𝘀. What's the difference?
Both use AI agents. Both get code in minutes. Most people think pros just know more. They don't. They have different habits, and some of them sound backwards.
❗️ 𝗖𝗹𝗮𝗿𝗶𝗳𝘆𝗶𝗻𝗴 𝗼𝗻𝗲 𝗱𝗲𝘁𝗮𝗶𝗹: we took
𝗔𝗜 𝗺𝗮𝗱𝗲 𝘂𝘀 𝗳𝗲𝗲𝗹 𝗹𝗶𝗸𝗲 𝘄𝗲 𝗱𝗶𝗱 𝗺𝗼𝗿𝗲. 𝗕𝘂𝘁 𝘁𝗵𝗲 𝗿𝗲𝘀𝘂𝗹𝘁𝘀 𝘄𝗲𝗿𝗲 𝘁𝗵𝗲 𝗼𝗽𝗽𝗼𝘀𝗶𝘁𝗲.
Most of us are sure AI makes us faster. In 2025, one study actually checked.
METR gave 16 experienced developers 246 real tasks from their own projects. A c
𝗪𝗼𝗿𝗸𝗳𝗹𝗼𝘄𝘀 vs. 𝗔𝗴𝗲𝗻𝘁𝘀. What's the difference?
A lot of people call anything with an LLM and a few tools an "agent." Most of the time, what they built is a workflow. Let's simplify.
❗️ 𝗖𝗹𝗮𝗿𝗶𝗳𝘆𝗶𝗻𝗴 𝗼𝗻𝗲 𝗱𝗲𝘁𝗮𝗶𝗹: the difference is who decides the next
𝗣𝗿𝗼𝗺𝗽𝘁 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 vs. 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴. What's the difference?
A lot of people use these two words for the same thing. They are not the same job. Let's simplify.
❗️ 𝗖𝗹𝗮𝗿𝗶𝗳𝘆𝗶𝗻𝗴 𝗼𝗻𝗲 𝗱𝗲𝘁𝗮𝗶𝗹: the context window is everything
𝗝𝗦𝗢𝗡 𝗺𝗼𝗱𝗲 vs. 𝗦𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲𝗱 𝗼𝘂𝘁𝗽𝘂𝘁𝘀 vs. 𝗧𝗼𝗼𝗹 𝗰𝗮𝗹𝗹𝗶𝗻𝗴. What's the difference?
A lot of people think these are the same thing. All three give you JSON back from the model. Let's simplify.
❗️ 𝗖𝗹𝗮𝗿𝗶𝗳𝘆𝗶𝗻𝗴 𝗼𝗻𝗲 𝗱𝗲𝘁𝗮𝗶𝗹: the real diffe
𝗖𝗟𝗔𝗨𝗗𝗘.𝗺𝗱 vs. 𝗦𝗸𝗶𝗹𝗹𝘀 vs. 𝗠𝗖𝗣. What's the difference?
A lot of people put everything in CLAUDE.md. Then Claude starts ignoring half of it. Let's simplify.
❗️ 𝗖𝗹𝗮𝗿𝗶𝗳𝘆𝗶𝗻𝗴 𝗼𝗻𝗲 𝗱𝗲𝘁𝗮𝗶𝗹: all three give Claude context (the text it reads
𝟑 𝐂𝐥𝐚𝐮𝐝𝐞 𝐡𝐚𝐛𝐢𝐭𝐬 𝐭𝐡𝐚𝐭 𝐛𝐚𝐜𝐤𝐟𝐢𝐫𝐞 𝐨𝐧 𝐲𝐨𝐮
1/ Starting every task in plan mode
We've all been taught to start every task in plan mode. But as frontier models improve, plan mode is getting less necessary. Even Thariq from Anthropic's Claude Code team is th