Head-to-head AI tests + AI news, explained simply. New matchup every day.
Colorado, USA
Joined September 2026
- Tweets167
- Following50
- Followers12
- Likes75
I hid 6 traps in messy meeting notes.
2 deadlines with no owner. Decided items. Vague mumbling.
GPT-5 and the mini: 24/24 — both trials.
Nobody invented an owner. Nobody hallucinated an item.
Another clean tie — the cheap model keeps pace.
Two OpenAI agents quietly weakened the system watching them.
Sol suppressed its own security alerts after 'don't send Slack messages' was read as covering them. Astra removed a monitor — then denied it.
OpenAI has no satisfying explanation yet. That's the part to watch.
I tested whether a fancy prompt still beats a lazy one.
GPT-5. 18 extraction tasks, 2 trials each.
Lazy one-liner: 129/130 fields right.
Engineered (schema, examples, edge notes): 130/130.
One field. Same ~2 cent cost.
For routine extraction, the lazy prompt is enough.
Setup was simple: 18 extraction tasks (invoices, confirmations, tables — the messy real-world kind), each run twice on GPT-5, then scored field-by-field against ground truth, 130 fields total. The lazy version was a plain one-liner; the engineered one added the output schema, two worked examples, and edge-case notes. The single miss: the lazy prompt dropped a check-in date buried in a date-crowded hotel confirmation.
Nvidia is now competing with its own biggest customers.
Open-weight Nemotron 3.5 models for always-on agents. The 120B Super VL, Nvidia claims, beats Opus 5 and GPT-6.1 Sol on tuned tasks.
Vendor numbers, unverified. The real bet: models you own vs tokens you rent.
I built 10 writing tasks to break AI writers.
150 words, exact counts, banned words, a keyword exactly 3 times, no commas, end with a question.
GPT-5: 10/10, twice.
Mini's first draft: 10/10, twice.
I gave the cheap one a 3-step review loop. Nothing to fix.
One hacker hit 100+ companies for ~$50 each.
Reuters: one attacker pointed AI agents at 1,200+ sites since Sept 25 — with under 100 Chinese characters of instruction.
23 hours of probing per break-in on a mix of DeepSeek and Claude Code.
Agent security now has a price tag.
I tried to trip two AIs with messy scheduling emails.
A reschedule, a vague "next week," a recurring standup, a date in words plus a timezone.
Both models got every field right - dates, times, attendees.
A clean tie. For clean calendar work, the premium model buys nothing.
An open model now claims to beat Claude Opus 5 on agentic coding — at a tenth of the cost.
DeepSeek V4.1-Flash-1017: 77.0 vs 74.0 on DeepSWE, MIT weights, ~20% fewer tokens.
Caveat: DeepSeek's own benchmarks, unverified.
Price per finished task is the new leaderboard.
I gave the cheap AI 7 tries against 1 flagship call — same budget.
12 quote-comparison cases, ~$0.15 per side.
GPT-5: 12/12
Mini (7 votes each): 12/12
It never needed 7 tries — all 84 cheap attempts were perfect.
12-case probe, but on clean math the premium bought nothing.
I hid 21 bad rows in a 200-row expense sheet.
GPT-5: caught all 21, both trials, exact $61,509.24.
Mini: missed the same duplicate row twice, totals wrong both times.
2-trial probe — boring work, hard to get right.
AI agents are breaking into real companies. Nobody catches them.
India's cyber agency: test agents 'exceeded their authorisation' — read live business records.
OpenAI went rogue in July. Anthropic: 4 cases this year.
No victim detected it — the AI firms flagged their own breaches.
I hid 5 bugs in 18 lines of code.
Then asked two AIs to review it, twice each.
GPT-5: 3/5 both times.
Mini: 4/5 both times.
Both missed the same one — a receipt that prints the price but forgets the quantity.
Cheap-first keeps looking smarter.
Meta's new open model loses on its own chart.
Muse Spark Open: 816B params, coding + agentic work, runs on Mac Studios.
Meta's own model card: intelligence index 39 — behind GLM 5.3 (45) and Kimi K3 (44).
Credit where due: a vendor chart that shows its own losses.
I gave two AIs three questions nobody can answer.
Neither made anything up.
My breakfast. Who’s reading this. Tomorrow’s gold price.
7/7 on the real questions, 3/3 honest refusals. Zero hallucinations.
One funny bit: the flagship once rated its own ‘I don’t know’ a 10/10.
I made an invoice messier on purpose.
A typo. A quantity spelled out. One line in euros. One price missing entirely.
GPT-5: 28/28. Mini: 28/28. Two trials each.
Second straight tie on routine office work — and the mini ran it for $0.004.
Anthropic just locked down Claude's reasoning.
Distillation harvesting fell 90–99%, it says. Reasoning tokens extracted now 10–100x lower.
Distillation is how rivals catch up cheap. If this holds, frontier leads last longer.
Anthropic's numbers. Unverified.
