SlopCodeBench is the most relevant coding agent benchmark left--the top score is 16% vs. 50-75% on other coding benchmarks like Terminal-Bench, FrontierCode, and DeepSWE summary of how it works: an agent receives a spec for a small program, like a backup scheduler or a mock HTTP server. the agent implements it, then the agent receives a new spec and has to extend the program. this repeats 3-8 times, and with each new spec there are hidden tests that check the new feature and all the old features. there are a few reasons why I really like it: 1. it comes far closer to simulating real, long-horizon work with new and updating requirements, which is what it’s actually like to work on a codebase vs. solving a single isolated task 2. code coherence, maintainability, and extendability naturally matter more 3. it’s significantly less saturated than the other coding agent benchmarks full results in the image (credit to @dexhorthy for the benchmark run + charts), the “core” number measures whether the main tests for the new feature pass; the “strict” number is the headline and measures whether every test passes, including regression tests for all earlier features astra and sol are at the top (interestingly 6.1 is slightly worse than 6 although likely just noise), but given how low the scores are overall + how small the spread is, i don’t think it’s super meaningful yet in terms of determining which model you should use--opus 5.5 is still my daily driver, and i’m kind of surprised it’s not higher the big caveat imo is that these runs all used the base harness with no customization. i personally think having a couple of core skills + agents.md instructions (planning, verification, etc) is necessary and improves performance, and that improvement is much more likely to show on SlopCodeBench than the other benchmarks my prediction is that custom context + skills would improve performance by 5 percentage points or more and (hopefully) result in a ranking that’s a better representation of how good different models are to actually use for engineering work do you think the improvement from proper skills/verification would be more or less than 5%?