San Francisco, CA
Joined July 2010
- Tweets11.8K
- Following161
- Followers56.4K
- Likes35.8K
Pinned Tweet
How to effectively debug with agents when you no longer read or understand the code
In this post, I propose a debugging framework for agents that doesn’t rely on human knowledge of code to get to the right “root cause”
The TL;DR are the screenshots.
Link to post in the reply.
Unfinished coding side projects are now way easier to complete than unfinished blog posts.
I’m perfectly OK with AI writing my code, even the docs.
But I flinch at the idea of AI writing my blog posts, which feels like an egregious violation of a fundamental ethnical contract.
I started this series on testing nearly a decade ago, but never published the final part copyconstruct.medium.com/tes…
There’ve been so many great advances in the storage space in the ~7y I feel this is worth completing.
Have a draft from years ago I hope I can wrap up in early 2027.
I’m literally that weirdo who wrote (all by hand) a 15K word blog post on:
- why one cant treat models as oracles
- how models gaslight you all the time
- what it takes to build confidence in the model’s output
copyconstruct.medium.com/how…
So yeah sure, I believe in magical woo-woo
To create my claude.md, I had it read:
- my favorite programming books + blogs
- the best oss projects
- the code of the engineers I respect the most
Claude (when given proper instructions, constraints, and context) now writes better code than me, almost always.
the coding agent haters are getting dunning-kruger'd so hard right now
do you really think that your engineering ability is better than the models at this point?
it is very very likely that fable is better than you in every branch of software
it has been trained on all the code of the internet, it knows almost every trick in the book, all the design patterns of every domain, every single low-level detail, every bit-twiddling hack, it's read every single blog post from every cracked engineer that ever existed
do you really think you can beat it? do you really think you can be more creative than it?
unless you are 99th percentile expert in some niche domain of software, it simply is not possible. sorry.
Many moons ago, I wrote about “post-commit” reviews.
In the agentic era, a workflow I find extremely useful is … *pre-commit* agentic reviews.
For large changes, the review agents take 10x the time it takes the coding agent to create a patch, but is totally worth it, IMO.
Blogged: Post commit reviews
Where I question if pre-commit mandatory code reviews are truly needed and what a post-commit review world would look like.
Caveat: post-commit review doesn’t necessarily mean post-deploy review.
bit.ly/2Oid8jG
It’s disappointing to see people say all code is going to be slop and we should be ok with it, because humans weren’t writing the best code either.
Quality and reliability should be higher (as quantitatively measured), not lower.
Why’re we lowering the bar for ourselves? 😔
Cindy Sridharan retweeted
Resharing this and urging everyone to read it for several reasons.
1. Everything Cindy writes is fantastic and serious engineering discourse
2. The examples here are concrete and clear and most importantly production services with real world consequences
It’s criminally long and took over a month to write.
I thought of making it shorter, but figured anyone who might care to read is just going to ask an LLM for a summary, so might as well leave it as it is.
copyconstruct.medium.com/how…
I’ve found that agents are great at debugging any problem when provided with this two-tier framework for troubleshooting.
This requires enforcing discipline and imposing constraints, rather than give agents a free rein or vague end goals, but works better than unguided “loops”.
Cindy Sridharan retweeted
Today @bitdriftio is really excited to open source blob-stream: a Kafka alternative for no fuss, low cost high volume streaming. I wrote in depth about what blob-stream is, why I made it, and how I made it.
blog.bitdrift.io/post/blob-s…
People talk about the diminishing returns of writing unit tests in the agentic era, but what’s even less useful? Comments.
As a rule, drop all comments. If something is important enough to need a comment, make an exception for it and audit the comment so it actually makes sense.
I need a personal agent for work where it:
- triages all incoming bugs
- manages slack comms
- gives me a summary and classification of all emails (8000+ unread), design docs, SEVs etc
I don’t need an agent for my personal life, where I just don’t deal with anything like this.
Here’s an example where Twitter Engineering was way ahead of the curve.
Many companies didn’t have to deal with “Twitter scale”, but agentic architectures are fast changing the scale even small orgs are expected to handle re stateful load balancing, orchestration, storage etc.
This post on the evolution of client side (distributed w/o coordination) load balancing at Twitter is 🤯
- [service] "mesh" topology + P2C has cons
- the "random aperture" approach which scales unfairly
- "deterministic random aperture" scalable + fair
bit.ly/2FvTH2u
Ball-knowers knew all along that Twitter wasn’t just going to collapse after Elon fired 75% of the staff, because Twitter 1.0 had some of the best infrastructure engineers working on it, and when you get that calibre of people building your foundations, it lasts.
With agentic coding, even in the best case scenarios (where you read every line of generated code), there’s something we’ve lost, perhaps irrevocably.
That has a real cost, & not to mention, second and third order effects, that “software factories” etc don’t necessarily solve.
Writing a post about “how to effectively debug code when you no longer fully understand it”.
This is real bug I had to fix in ~9K LOC (100% AI generated).
None of the tactics I started with here helped “root cause” the issue, lol. Had to take a different approach to isolate it.
One thing I don’t see many talk about is how coding agents can often *amplify* weaknesses, both individual and organizational.
I think of this as the “empty calories” problem, which is exacerbated when “hyperproductivity” originates from your team’s “lowest common denominator”.
The IDE/editor might be dead, but it helps to occasionally open one and read some code.
Often, what “looked OK” in isolation when an agent wrote a patch can end up looking *very* different from more of a “panoramic view”, with misshapen and incoherent APIs, abstractions.
Incidentally, this proves that with AI you can now solve problems *way* out of your wheelhouse or your “knowledge/skill” level, if you persevere enough and can get creative with how you approach debugging.
It’s a different skill set than coding or even traditional debugging tho.
I previously called this an “emergent” skill set.
The people yapping about how you “don’t need to understand the code anymore” are NOT telling you how to sharpen or even hone this new skillset, where you *can* debug problems out of your comfort zone, methodically with AI.