CTO @sherlockdefi. Hand-wrote systems where one bug meant an 8-figure loss. Trying to understand where AI takes software engineering.
Amsterdam, The Netherlands
Joined February 2020
- Tweets618
- Following287
- Followers647
- Likes2K
I believe that 2027 will be the end of the frontier model releases, and labs will start to focus on vertical integration of their internal models in the economy.
Evert retweeted
I still believe a correct framing on LLM usage is that they can get you 80% of the way upfront, but that last 20% starts to become impossible to overcome. The proper way to use an LLM is to do the difficult manual 20% upfront. Once you have some decent structure, decent API surface area, decent data structures laid out, decent base libraries. Then you can largely prompt your way through the last 80%. While the first 20% which you did manually, intentionally and fully comprehend. That 20% essentially becomes part of your development harness, almost a prompt itself. Where you can steer its overall architecture, or data design, or other critical pieces by keeping a grip on that first manually done 20%.
Something real concerning with this. Knowing what to do in that first 20% requires advanced knowledge of the problem domain you are going to deal with. It is a staff or principal level engineer domain. Which most simply can't do.
Which means most people are going to run the gauntlet heavily relying on prompts, and are going to cover some incredibly impressive ground very quickly. Because 80% of the way is quite a long ways there. It looks like it could almost be done. But then they will get stuck in that last 20% and will never be able to make it perfectly polished. Nor perfectly sable. Nor remove all the odd quirks. Nor overcome the last pieces of performance or memory issues. It will basically hit a plateau where they lose control of it. Or it becomes unwieldy to take it in a new complex direction, or add some novel complex functionality to it. It will be very hard to tell from the outside if any given company or project is aimed at getting stuck in that last 20%. As they could appear to be kicking some major butt in that first 80%. But then it will reach a plateau where they can no longer do anything substantial without breaking something else, or seem unable to fix every little quirk or issue. A lot of companies will probably get stuck here. With dozens of very impressive things 80% of the way, but seem unable to get over the last final 20% to really perfect it and polish it.
Gave Astra a medium size task, came back to an usage limit error and a PR where it included a fork of an open source project 馃ぃ
Codex with 5.6-sol is annoying. Introduces way too much code, and when I ask it to simplify, the harness rejects the changes as it "weakens the trust boundary"
Slop machine
Prediction: someone builds the harness that stops coding agents from drifting. Intent capture, invariants, knowing when a refactor is due. Then the labs train the next generation inside it, and the scaffolding disappears into the weights. Same as chain of thought.
Adding devs to a late project makes it later. Coordinating many humans in one codebase was one of the hardest management problems in software. Agents didn't remove that problem. They removed the salary, so now everyone gets to rediscover it at 100x the speed.
Two codebases both satisfy your requirements today. Which one is better? The one that ships the next 1,000 features fastest. For decades that meant clean code and good architecture. With agents writing the code, I think it comes down to how cheaply you can verify each change.
2025: hand the AI a function and it writes it well. 2026: hand it a feature and it implements it well. The endpoint is handing it the whole product. I think the blocker for that last step isn't code generation. It's that nobody wrote down what the software must never break.
The usual answer to AI code quality problems is that the next model fixes it. I don't think it does.
A better model writes faster and reads a codebase faster. It still doesn't know what the system is supposed to guarantee, because that isn't in the code.
The problem with agents and context isn't that nothing is written down. Most of it is. Specs, tickets, old discussions.
It's that nothing says which of those still apply, and none of it is in front of the agent when it makes the change.
When do you hire another dev, and when do you just spend more on compute?
I can't answer that cleanly yet, and I think it's becoming a real budgeting question.
If an agent can rebuild your product from a good enough description, the code stops being the asset. The description is the asset. Most companies have this exactly backwards: the code is exact, but nobody could hand you the description.
An agent can rebuild the map of a codebase in minutes. What the parts are, how they connect. Each model generation does this better.
What it can't rebuild is the intent. Code shows what the system does. It doesn't show what it must do.
It can be cracked with current models. The missing piece isn't the infra in [2], it's capturing intent and invariants. A factory that doesn't know what must never break just produces drift at scale.
[1] software factories are super real but we need to be realistic. the factory aspects haven鈥檛 been cracked yet, the [3] innovators are toying around with discovering practices and pieces.
if someone is selling you a factory rn and they aren鈥檛 in the super small cohort (ie. they are likely startups created in last six months!) or circa <10 people in the who鈥檚-who, who have been trying (and realistically failing) over the last two years.
they are selling bullshit.
it isn鈥檛 cracked but it鈥檚 a puzzle that鈥檚 being solved daily. the best practices right now are to build abstractions/pieces needed which entails solving [2] non-agent topics.
[1] almost lights out.
[2] sandboxing, monorepo, reproducible builds, ci/cd, identity/secret management, smashing corporate friction in the realm of devex for agents.
[3] at home, in your homelab, you should be cracking on this problem space in your free time if you want a super fast promotion and red carpet service at your next interview. highest roi you can have right now is learning the entire damn stack, automating it and showing it at your next interview/doing a recorded talk at a meetup.
An agent sees what the code does. It doesn't have a picture of all the ways the code is expected to be used.
So it makes a change that's correct against everything it can see, and wrong against everything it can't.
Wrote down 22 system invariants in english, then spent the weekend having a model try to break every one.
It found 11 real issues. One was critical, caught before it shipped.
The elite of the elite
Today, @blackthornxyz emerges from stealth.
The most exceptional protocols deserve to work with the most elite auditors.
Blackthorn's 10 founding auditors have won over 120 audit contests.
Blackthorn's limited protocol members represent $160Bn of TVL.
Evert retweeted
Today, @blackthornxyz emerges from stealth.
The most exceptional protocols deserve to work with the most elite auditors.
Blackthorn's 10 founding auditors have won over 120 audit contests.
Blackthorn's limited protocol members represent $160Bn of TVL.