fiddler & engineer
New York, NY
Joined November 2010
- Tweets2.3K
- Following5.1K
- Followers1.4K
- Likes32K
Dave Kerr retweeted
Google’s Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5, and second only to Claude Opus 5.5 on the Artificial Analysis Coding Agent Index
Gemini 4 Carbon scores 59 on Intelligence Index v5, driven by leading performance in complex reasoning, knowledge, and scientific tasks. It also has large gains in coding and knowledge work, particularly when compared to other Gemini models, but it is not leading in all these areas.
As with launch of Gemini 4 Argon, Gemini 4 Carbon is being rolled out to selected users and is not yet publicly available. Google is offering a 50% launch discount, with no confirmed end date. All our charts and figures use standard pricing for both Gemini 4 Carbon and Argon.
Key results for Gemini 4 Carbon with high reasoning:
➤ Gemini 4 Carbon leads the Intelligence Index v5, with the #1 score in 4 of the 9 Index evaluations: HLE-Diamond, Terminal-Bench-Science, AA-Omniscience, AutomationBench-AA. Carbon makes substantial gains elsewhere but is not leading in coding and knowledge work
➤ Behind only Claude Opus 5.5 on coding agents: With Antigravity CLI, Gemini 4 Carbon (high) scores 59 only 3 points behind Claude Code with Claude Opus 5.5 (max, 62) on the Artificial Analysis Coding Agent Index. Carbon ranks second on both Terminal-Bench 4.0 (64%) and FrontierCode v1.1 (51%), behind Claude Opus 5.5 (max) at 56%. Google has historically underperformed in this area and so this improvement stands out
➤ Improved agentic knowledge work, but not yet leading: On AA-Briefcase, Gemini 4 Carbon (high) scores 1609 Elo, a 126-point improvement over Gemini 4 Argon (high, 1483), but still behind models including Grok 4.7 (xhigh, 1638), Muse Wonder (max, 1659) and Kimi K3.1 Pro (1678). The largest gain comes from Presentation, up 293 Elo to 1598, alongside a 120-point improvement in Analytical Quality to 1671
➤ Lower Cost per Task than Claude Opus 5.5 for comparable overall intelligence: Gemini 4 Carbon costs $7.76 per Intelligence Index task, ~23% less than Claude Opus 5.5 (max, $10.04) but ~1.2x GPT-6 Astra (max, $6.25). It uses 123k output tokens per task, 29% fewer than Claude Opus 5.5 (max, 173k). At the same $4/$20 standard pricing, its Cost per Task is in line with Gemini 4 Argon ($7.68), while scoring 9 points higher on the Intelligence Index
➤ Leading AA-Omniscience v1.1 score, with low hallucination: Gemini 4 Carbon scores 54 on our recently updated AA-Omniscience v1.1, 6 points ahead of Claude Opus 5.5 (max, 48) and 8 points ahead of Gemini 4 Argon (46). Accuracy (% correct) rises 10 points over Argon to 58%, while its hallucination rate rises from 4% to 11%. This is still the second lowest of any model, behind only Argon, compared with 50% for both Claude Opus 5.5 (max) and GPT-6 Astra (max)
Key model details:
➤ Context Window: 1M tokens
➤ Multimodality: Text, image, video, and speech input, with text output
➤ Pricing: $4/$20 per 1M input/output tokens, the same as Gemini 4 Argon’s standard pricing, with a 50% discount. Cached input tokens receive a 95% discount ($0.20 per 1M)
Congratulations to the @GoogleDeepMind team!
Dave Kerr retweeted
I'm starting to sense a divide in the tech industry:
- People thriving on autonomy who are reinventing how they work and what tools they use with AI: LOTS of work, but they're enjoying it and getting very efficient
- People used to being mentored / trained / or simply told what to do and how. Frustrated with the lack of guidance from management on what to do with AI; also frustrated by seeing some of their peers thriving; increasingly bitter about AI and the state of the industry, as a whole
Neither group seems to be aware that there's another group, or have empathy for the other group
Dave Kerr retweeted
Today, in collaboration with Anthropic, we’re excited to share rapid initial progress towards an antibody treatment for food allergy, which affects over 200 million people worldwide, discovered with the help of Claude.
At Divergence Labs, we believe that AI can unlock the next generation of antibody therapeutics with more effective mechanisms than before. Such mechanisms are urgently needed in food allergy, which involve overcoming one of the tightest affinity protein pairs in immunology, IgE and its receptor FcεRI, on mast cells.
Treatment with Xolair—the only approved antibody for food allergy—sequesters circulating IgE, but mast cells remain sensitized for weeks to months due to the extraordinarily slow dissociation of IgE from FcεRI.
We envision a treatment that actively pulls IgE off of FcεRI, without waiting for natural dissociation. Like picking a lock, this requires the right combination of geometry and force.
Our research team, working with Claude, ideated on mechanisms to accomplish this disruption, and converged on an antibody format with two arms with different functions and epitopes. Within one month, we designed, synthesized, and experimentally confirmed the biophysical function of the two arms, which we are now assembling into full therapeutic formats.
Dave Kerr retweeted
Not at all. 4 years is very short. Social media glorifies shipping speed but real, durable ideas all take a decade or more. Very few quick wins that aren't just flash in the pan. Dig in deep and expect the good ideas to be full time work for a long, long time.
Dave Kerr retweeted
feels like there needs to be a new way to distribute opensource software such that people can modify via agents but also get upstream automatically merged in
coding agents can do this but they won’t do it automatically
Dave Kerr retweeted
Biggest opportunity now as a software engineer is clear: master parallelism → outsize scale of output. Value creation in software has always come from leverage: small input, outsized output.
Parallelism-maxxing is not extremely quantifiable because it's so new. The spectrum runs between people managing a couple of agents on their local machine to those running an infinite number of agents + computers at a time than can be spun up and down on demand. You want to be at the far end of this spectrum, and you should be ROI-maxxing every token spent.
If you can do this and prove it, you will have an infinite number of opportunities available.
If you're still trying to do this locally you are at an objective disadvantage, equivalent of hosting a startup on a server in your closet vs using AWS. And if you're spending your time building and maintaining infrastructure that gives you no competitive edge, the AWS analogy applies in the question "why are you wasting your time doing undifferentiated heavy lifting instead of building unique customer-facing features?".
Dave Kerr retweeted
Caring is the moat, you should feel visceral pain when things are bad or broken and stop at nothing to fix them
Dave Kerr retweeted
Effective Immediately, all organizations using PHP for their systems are advised to be prepared for an update addressing two critical vulnerabilities. PHP is widely used for web applications. These vulnerabilities often provide a direct path to the server's file system or database. If an immediate update is not feasible, it is recommend that organizations take the system offline until it can be fully mitigated. NSA will follow up shortly with more specific guidance.
Dave Kerr retweeted
Getting people to care about what you built remains arguably the hardest part of building software
Dave Kerr retweeted
volume does matter
look, i get it. if you were an engineer before agents came along, you're rightfully skeptical about lines of code and number of PRs being used as a metric for any kind of productivity. and i agree! or at least, i used to. for ease of explanation i will talk about this from the perspective as being an engineer on a large team of at least 50.
volume didn't matter before because we focused on impact. you could have high impact even with a few PRs: for example you could make a one line config change and save the company millions of dollars.
but before agents, the times where i saw volume come into the conversation were for the outliers. on the negative end you might have someone who is struggling to perform, on the other end you might have someone who was a "coding machine" archetype. a coding machine was someone who could just lock themselves in a room and emerge with a stack of PRs that solve a huge number of problems. so volume did matter, but only at the tail ends of the distribution, and combined with impact.
if you've ever worked at a large tech company before, you'll get what i mean. the thing about agents is that big tech co problems are now small-medium co problems as well.
how do agents change this calculus? well, everyone now has the ability to become a coding machine. you have incredibly capable frontier models that can write code better than any of us, at a rate much faster than we can. in this world, if you're still producing the same number of PRs as before, why wouldn't you pause to wonder why you're not being more productive? alien intelligence is here, and yet you can't outship a human coding machine?
this brings me back to why i keep talking about trust (xitter.cf/poteto/status/21020504…). if you haven't put in the work to trust your agent's output, it is very difficult to scale up your productivity.
and yes, this approach does require more tokens, but i think about this in terms of cost per intelligence. before agents, this cost was very high - you needed to hire many software engineers with high 6 figure salaries to do the work. tokens are still expensive today, but are likely to get cheaper over time (xitter.cf/theo/status/2107924138…) factoring in the cost of intelligence. what looks exorbitant today will likely be affordable in 6 moths to a year. a single engineer with agents can do the work of tens if not hundreds of engineers, at a fraction of the cost.
the reality that i don't think has hit yet is that job of the software engineer has truly changed. our job is not merely to produce software any longer (well tbh it was never only about the code, but bear with me for the sake of the explanation), but the machine that writes the software. this idea of a software factory, or a Michelin kitchen as i like to call it, is still a topic of research. and it's the type of stuff i like to share, not to flex, but to show you what's possible when you put a lot of rigor into using agents at scale
Dave Kerr retweeted
This isn't your typical acquisition blog post. I'm really excited not just that @rough__sea, @piscisaureus, and friends are joining, but also what they plan to work on: Making workerd a first-class, self-hostable open source runtime. Read for more:
blog.cloudflare.com/deno-joi…
Dave Kerr retweeted
new in mise: tools that keep themselves up to date.
set auto_update on a global tool and mise upgrades it right before it runs. optionally run a daemon to do upgrades in the background.
mise.jdx.dev/dev-tools/#auto…
Dave Kerr retweeted
I am once again, with @poteto, advocating you to do the hardest thing possible justinjackson.ca/hard-thing
Dave Kerr retweeted
The people wanted it, here it is.
Full disk search, in milliseconds. 17,000x faster than Finder.
github.com/noahdunnagan/fsea…
Dave Kerr retweeted
I think I found a planet nobody knew existed.
It's been hiding in NASA telescope data for seven years.
At the end of this thread, you can fly to it yourself.
Dave Kerr retweeted
Today AWS open sourced Strands Box, a local sandbox for AI agents. @MarcJBrooker wrote a super in-depth post on the design tradeoffs, how OS-level containment and Dogwood enable richer policies, and AWS’s investment in agent safety. Def worth a read:
strandsagents.com/blog/stran…
Dave Kerr retweeted
Anthropic has released Claude Haiku 5.5, scoring 43 on the Artificial Analysis Intelligence Index - up 26 points one year after the last Haiku release
Haiku 5.5 is the first Haiku model with Anthropic’s effort settings and adaptive thinking, and Anthropic has introduced tiered pricing.
Haiku 5.5 is cheaper than its predecessor - it costs $0.10/$0.50 per 1M input/output tokens for prompts up to 100k tokens (the same as GPT-6 Luna and 10% of the previous Haiku model). However, this pricing rises 5x to $0.50/$2.50 above 100k. The site does not yet reflect tiered pricing, so provisional cost figures for Haiku 5.5 do not include the step up cost. We are working on support and will follow up with Cost per Task coverage soon.
Key takeaways:
➤ Leading small-class model performance: At max effort Haiku 5.5 sits slightly ahead of models such as GLM-5.3 Flash (42), Gemini 3.8 Flash (41) and GPT-6 Luna (38). Its score is comparable to Kimi K3 (44), a 2.8T parameter open weights model, and trails Claude Sonnet 5.5 (max, 56) by 13 points
➤ Heavy token use compared to GPT-6 Luna: Haiku 5.5 (max) uses ~162k output tokens per Intelligence Index task, ~3x GPT-6 Luna (max, ~50k). Moving from xhigh to max adds 2 points for ~1.8x the tokens. At similar intelligence it also uses more tokens than GPT-6 Luna: Haiku 5.5 (high) scores 38 with ~55k tokens per task against 38 with ~50k for Luna (max), and the gap widens at lower effort settings
➤ Highly capable at agentic knowledge work: on AA-Briefcase, our private evaluation for realistic knowledge work tasks, Haiku 5.5 (max) reaches 1578 Elo, ahead of models including Kimi K3 and GLM-5.3, and comparable to Muse Spark 1.3 (max)
➤ Improvements on terminal use: on Terminal-Bench 4.0 it scores 33%, up from 0% for Haiku 4.5. This is level with GLM-5.3 Flash, and ahead of Gemini 3.8 Flash (20%) and GPT-6 Luna (13%)
➤ Lower factual knowledge, but relatively low hallucinations: as expected for a smaller-class model, Haiku 5.5 has lower factual knowledge than its siblings. AA-Omniscience accuracy is 36%, against 55% for Gemini 3.8 Flash and 44% for GPT-6 Luna, but this is partly driven by more willingness to admit when it doesn’t know - its hallucination rate is lower, at 40% against 55% and 77%
➤ AutomationBench-AA result likely understated: Haiku 5.5 scores 35%, against 53–60% for GPT-6 Luna, Gemini 3.8 Flash and GLM-5.3 Flash. During pre-release testing, a safety refusal issue caused the model to over-refuse. Anthropic is working on resolving this - we will re-run this evaluation with the fix, and expect this score to rise
Other model details:
➤ Context window: 1 million tokens, up from 200k for Claude 4.5 Haiku
➤ Pricing: $0.10/$0.50 per 1M input/output tokens up to 100k tokens, $0.50/$2.50 above. Cache reads $0.01 ($0.05 above 100k), 5 minute cache writes $0.125 ($0.625 above 100k)
➤ Multimodality: Text and image input, with text output
Dave Kerr retweeted
I've published a new, general-purpose terminal specification that gives any program a way to tell the terminal what it's doing: idle, working, waiting, finished, or failed, and why. It is easy to implement on both sides. I've written more about why: mitchellh.com/writing/progra…
The problem this solves is generic and applies in compelling ways to all sorts of use cases (e.g. package managers, build tools), but it really rears a hideous head when it comes to AI and the "agentic inbox" problem.
I found over 250 different agent orchestrators (with various goals) that all individually implement a heuristic based approach to detecting whether things like Claude Code are working, blocked, done, etc.
If you look at @herdrdev's commit history, you can see around 10 compatibility changes to detect Claude Code within just the past 3 months. (This isn't a dig at Herdr! They're doing great work with what they have, and everyone else is doing the same thing).
Heuristics are not the way. And over-indexing on agentic use cases isn't the way, because this problem is really interesting for other tools too: Homebrew, Terraform, Cargo, etc. etc.
Proprietary protocols and out of band APIs are not the way. They have an O(N) integration problem and require extra work to work over SSH or within local VMs/containers.
The program status specification (OSC 7501) solves this in a reliable, deterministic way that is general-purpose and can be easily adopted by the entire industry.
Read more: mitchellh.com/writing/progra…