We ended up rolling out a deterministic hook that slaps Claude on the wrist with non-zero exit code and a little context injection on how we want our code comments (no comments is the best). Result on the screenshot - we are back to pre-Opus 4.7 comment % in our PRs!
Prepping slides for my talk at SignalsConf – here's an interesting one – our PR revert rate has dropped 3x right after we rolled out our latest generation of the code review and approval agent!
We've been fighting with Claude generating lots of confusing code comments. Here's a timing correlation between Opus 4.8 rollout and the % of comment lines (2x bump!) – this is based on 42k merged PRs assessed by our code review bot. Anyone have any solutions that worked?
Inokentii Mykhailov retweeted
I wrote up a brief checklist of good vs bad skills on our blog today. Link below.
Webinar was great but I'm especially proud of us doing such a quick turnaround answering ALL the questions!
Another banger thread and blog post by @ryan_sherlock ! Ryan makes it look easy but it took about a decade of gruelling, tedious, diligent work to get to this level of operational excellence. I've been lucky to be on the hook for our Elasticsearch infra for a long time – learnt tons about distributed systems.
Super excited for this one 😍
Most engineering orgs are experimenting with AI. We bet the entire company on it, and hit our 2x productivity goal 3 months early.
Join us on May 19 for a live session where Fin CTO @darraghcurran, Principal Engineers @brian_scanlan and Kesha Mykhailov, and special guest @clairevo break down how Fin’s ~500-person R&D org:
• Shifted to agent-first engineering
• Got 95% of PRs running through Claude Code
• Cut defects by 55%
• Shipped 2x more product changes at nearly half the cycle time
You'll hear about the messy middle nobody talks about – uneven adoption, code quality dips, the human bottlenecks that mattered more than the technical ones, and what they'd do differently.
Register at the link in the replies.
You hear "coding is solved" a lot these days. Maybe. What's not solved yet is delivering massive product changes to your customers quickly and safely. An agentic coding tool will gladly create a thousand-line Pull Request on your behalf – but no human/bot review can properly de-risk those monsters. The only proven way to ship a large, risky diff is by splitting it into smaller, self-contained MVPs, uncovering all the unknown unknowns one by one, incrementally, in production – your only objective reality. Our ambitious goal is to create a positive incentive in the socio-technical system (Intercom's R&D) for folks to ship in smaller increments, faster and hence safer. Everyone will have a choice to either create a large PR and wait hours for a human review or ship things incrementally and benefit from streamlined reviews and auto-approvals from our agent. Here's the post with more details:
intercom.com/blog/ai-is-appr…
Every second matters! Amazing work by @_byroot !
With the new job, I didn't get much time or energy to blog lately, but I figured I'd use the various downtime while traveling to Ruby Kaigi to write about some of the things I worked on recently: byroot.github.io/ruby/perfor…
Pumping pressure into the system to surface real bottlenecks is the ONLY way to get your shipping factory modernised. AI code generation means nothing unless you set ambitious target to drive a real socio-technical change in your R&D. Hats off to @darraghcurran – true pioneer!
9 months ago we publicly committed to 2x the productivity of our R&D org at @intercom.
It was scary. It wasn't always clear we'd pull it off.
We hit it with 3 months to spare. In fact, looking back 16 months - we've 3x'd.
Here's what actually happened (with receipts): 🧵
This week at Intercom we hit over 19% of PRs auto-approved by our PR review agent based on Claude Code. Our ambitious goal is to get to 50+% by the end of this month. I'll spill all the details below and you decide yourself if we are out of our damned minds or onto something...
Only driving a cultural change will get us to 50+% auto-approval – creating a positive incentive to stop pushing massive unsafe AI slop that is harder to review (human or not). Instead ship product incrementally, faster and hence safer. Shipping is our heart-beat after all!