Principal Engineer in London, speaker, tabletop gamer, geek. Tattooed, pierced, and bearded. The 'guv' on @BrighterCommand. Also @ICooper@hachyderm.io He/him.
London
Joined January 2008
- Tweets38.3K
- Following1.7K
- Followers9.5K
- Likes31.4K
Pinned Tweet
I will be teaching my two-day course, Practical Messaging, at NDC Oslo this year. I suspect it will be the only public presentation this year.
ndcoslo.com/workshops/practi…
Trying to change my payment details on @github . GitHub appears to have never had a human being look at the MFA following a change, as you can't navigate to the confirm button on FF or Chrome.
Just bad. What's happened to GitHub.
Just created a skill to weed comments that agents have unnecessarily added in Brighter and execute refactorings they were hiding
ban and delete most if not all code comments!
agents love copying what already exists. a seemingly harmless comment explaining some temporary workaround is one of the worst viruses you can have in your codebase. like a weed it'll spread quickly as a tool for agents to quickly get their job done. before you know it, your codebase is overrun with hacks and stale comments. just one more workaround bro
you can use pstack's /no-comments skill as a weedkiller. it'll seek out useless comments while keeping the ones that actually should exist (like explaining a browser quirk), and figure out how to solve them on a first principles basis
here's a prompt you can try asking @bot:
create a project agent that uses /poteto-mode to spawn a /swarm to use /no-comments and identify useless code comments we can prune in my repo. come back with a high level plan of which of these workarounds would be best solved by refactoring and which we can straight up delete
x.ai/bot/plugin/9717366
Did they just reinvent Perforce P4?
New on our Engineering blog: Creating in-house source control.
In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale.
Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
Ian Cooper retweeted
Welcome to today's edition of "that number doesn't mean what you think it does"
The headline result ("don't have agents write tests") is unjustified for at least a few reasons:
1. The code with tests was literally higher scoring on average 😂 This is at least weak bayesian evidence *for* the idea that tests matter
2. The experiment is sloppy. I can guarantee that fuzzing some functions wil find bugs. Just comparing "write no tests" against the default isn't enough evidence to make the claim that tests don't matter
3. DeepSWE tasks are not necessarily representative of real tasks that you care about. And the scores are often not representative of this type of difference (for example, not all edge cases are tested, and testing is most useful for finding such edge cases)
4. There are other reasons to write tests. For example, code quality does not appear to have been measured at all. And unit tests serve as useful guards against spurious change ("mutation testing")
All of that said, it's likely true that just having an LLM write basic tests isn't super helpful.
But this is the same as requiring human software engineers to write tests--they will make tests without making the code better.
If you want better code, you have to actually care about that and think carefully about how to test it.
And yes, LLMs and coding agents can be very useful for that (if you're actually trying at all)
ok it's official... AI-written unit tests and integration tests are empirically proven to be unhelpful. you should tell your agents to stop writing tests by themselves
on the deepswe eval set, banning sonnet 5.5 high from writing any tests actually resulted in slightly higher success rate (non stat-sig), with less time and token spent (stat-sig). this is pretty hard empirical evidence
across the tests that were written by the "tests allowed" baseline arm, 65% of them were unit tests, 35% were integration tests, neither bucket resulted in any improvement compared to not writing any tests at all
i also picked a random sample of 44 tasks subset where i completely disabled executing even existing tests - it also did not affect success rate at all
i spot checked many tests written in the baseline arm, and my intuition is that most tests are simply a repetition of the implementation
agent-written tests do not add any value because both the implementation and the tests were simply the agent's interpretation of our intent. the tests aren't any more accurate than the implementation itself
i suspect we can still extract some value from unit tests and integration tests if we describe them ourselves when we believe we can articulate our intent better through test cases than through requirements. but i have not proven this yet
also worth noting, during deepswe eval the agent wrote almost no e2e tests (only 17, compared to 3000+ unit/integration tests written). so this analysis does not prove nor disprove the value of e2e tests - i will do another eval specifically for that
Ian Cooper retweeted
What a legend.
news.mit.edu/2026/margaret-h…
The original software factories movement was largely predicated on the idea that once you shifted to frameworks, all you needed was to code your domain and configure the framework.
Only issue was that DSLs authored by SWE used by domain experts sister work out.
In essence, the problem was that it’s a lot of work, to implement the DSL that products were crafted in, debug through the code generated via that DSL, etc. and it was debatable if that was faster/cheaper than an XP team writing the domain specific code in a 3GL.
Agents don't need to do manual memory management unless it is essential complexity (you can't get that performance from a GC'd runtime). Otherwise, it is accidental complexity. Accidental complexity just pollutes your context and costs you tokens.
We still need to do SWE, folks.
Hypothesis. You didn't know how to write unit tests (not at module boundaries, not in deep, narrow modules, and not by testing implementation details via mocks) or how to write better developer tests.
Because of poor training data, you have to show AI how to do TDD.
We deleted 800k lines of unit tests from the @sazabi monorepo earlier this week.
Three hypotheses:
1- Unit tests "lock-in" slop code by making it hard to change or remove
2- Unit tests slow down dev cycles by making your agent do more work locally and by increasing CI times
3- Unit tests waste money by making your agent generate more tokens and by adding useless CI runs
Will report back soon with the results!
This was **always** TDD, not the test method/class with mocks abomination that so many practiced and that polluted so many training sets
Will talk about it at NDC London: ndclondon.com/agenda/tdd-and…
Ian Cooper retweeted
A couple of my learnings reflected here:
- adversarial reviews (ideally with a 3rd agent as arbiter) at every step
- max coverage test suite, unit to e2e, no exceptions. TDD must be baked into agentic flows
- human review matters more for non-code stuff, particularly at the architecture and decision-making stages
- contextmaxxing in the repo: docs, ARDs, customer ICP descriptions, runbooks, schema commentary, all must live INSIDE the codebase, not just attached to it
- contextmaxxing in the process: track every agentic conversation + changes made (via Entire or similar) and attach to ticket/PR
- dev/testing DB content matters more than ever: dry runs, workflow simulations, edge case hunting, performance measuring needs to happen before things hit prod, seeds aren't dummy data; they need to be representative of prod, if not in volume, then in shape
- code that works best for agents doesn't look best to humans: humans need to think of themselves as validators, not proofreaders.
Comprehension is now a function of our tools, not a state of our minds. Weird, novel, and kinda scary, but things operate at speeds that typing letters into a text field can never catch up to again.
Ian Cooper retweeted
In the face of generative AI coding agents, the role of human-led architecture becomes even more important in the crafting of software-intensive systems.
All architecture is design, but not all design is architecture. Architecture represents the set of significant design decisions that shape the form and function of a system, where significant as measured by cost of change.
As such, remember that coding agents are reasonably good at transforming prompts into code, but they are not particularly good at making decisions that balance the static and dynamic forces on a system in order to yield a reasonably optimal solution: this is where human judgment, human understanding, human knowledge, and human experience shine.
Ian Cooper retweeted
xitter.cf/mizuki_izuna/status/21…
CAPCOM’s next-gen RE ENGINE work is very interesting.
They’re basically attacking several pain points at once: C++ complexity, long build/link times, memory corruption and leaks, awkward reflection, duplicated C++/C# type systems, expensive marshaling, IL2CPP debugging, GC pressure, and memory fragmentation.
The answer seems to be RE:Runtime + RE:C++.
RE:C++ is built around C++23, but with C#-like semantics layered on top: automatic memory management, generics, interfaces, boxing, exceptions, and access to the .NET standard library.
The really interesting part is the workflow.
CAPCOM is using C# as the frontend language for parts of the runtime. Code is written in C#, compiled into a normal .NET assembly, then RE:Build converts/decompiles that into RE:C++ and builds it into native libraries/executables.
So the rough pipeline is:
C# → .NET DLL → RE:C++ → native C++ binary
That means developers can work with C# syntax, tooling, libraries, reflection, attributes and unit testing, while the final game/runtime code still ends up native.
They’re also doing their own memory-management work underneath this. Instances are managed in 64 KB pages, with page reference counting plus cache and GC optimizations. The goal seems to be reducing the cost of managing huge numbers of objects while improving locality and avoiding some of the fragmentation problems they’ve had before.
This also makes CAPCOM’s REDox project much more interesting.
REDox is their open-source .NET serialization/data library, with JSON, CBOR, MessagePack-style structured data handling, lazy decoding, streaming, compact token storage, etc.
At first glance it just looks like a standalone .NET library.
In context, it looks much more like the kind of infrastructure they can write and test normally in C#, then feed into this RE:Runtime / RE:C++ pipeline and ship as native engine code.
#RE2026CAPCOM #CAPCOM
