Mark Müller (@mnmueller) profile picture

Co-founder and CTO at https://xitter.cf/t.co/l2sBCoRD0j. Working on monitoring and self-improvement for Software Factories. ML PhD from @the_sri_lab at @ETHZ

Zurich
Joined January 2021
Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)Photo by Mark Müller (@mnmueller)
New paper: Frontier Coding Agents Find Different Bugs Than Maintainers Fix In proactive bug finding, agents report fewer than 20% of the bugs humans fixed. But they report a lot of other bugs, most of which are valid. This points towards an important distributional mismatch.
10
3
4
35
6,553
This work was done with our M.Sc. student Marco Paina @logic_star_ai, in collaboration with @the_sri_lab.
3
97
1/10 Went through Anthropic’s S-1. Two things stand out: The unit economics look like normal software, at least for now Over 90% of their own code is written by Claude, and the risk factors read like a list of what goes wrong at that scale 🧵
2
2
3
241
9/10 Put together: generating code is cheap and getting cheaper, code per engineer is up 45x, agents can misreport “done”, and benchmarks don’t predict production. The bottleneck has moved from writing code to knowing whether it is correct.
1
2
26
10/10 This is the problem we work on at @LogicStarAI: monitoring and improving the real-world outcomes of coding agents run at scale.
3
18
9/10 Put together: generating code is cheap and getting cheaper, code per engineer is up 45x, agents can misreport “done”, and benchmarks don’t predict production. The bottleneck has moved from writing code to knowing whether it is correct.
1
5
10/10 This is the problem we work on at @LogicStarAI: monitoring and improving the real-world outcomes of coding agents run at scale.
4