the fact that frontier models are bad at chess (< 2200 elo) is a fascinating case study in jagged intelligence (which maybe provides ample copium about the post-agi economy) reasoning from first principles/public evidence: (1) it should be easy to post-train for chess elo -- super verifiable, tons of data (human games, self-play), free turn-level reward (stockfish), relatively cheap outcome-level signal (30-40 moves), easier credit-assignment than other tasks. (2) therefore, labs don't care about chess elo / aren't trying to post-train for it. and this has two implications: (a) post-training for core objectives (coding, math, science, knowledge work, rsi, etc.) still plateaus at ~95th percentile human ability on an ood task, even on a relatively simple/verifiable env like chess (b) there is no value/negative value for post-training for chess. either (i) it hurts core objectives/metrics (maybe leading too overthinking?), (ii) it takes compute/time away from core objectives. the core implication of (b) is that labs are/will prioritize pushing a narrow set of core objectives rather than smoothening the jagged frontier (even when its easy to do so, eg chess) (3) if it generalizes that (a) general models plateau relative to specialized models, (b) labs will prioritize performance on narrow set of core objectives it follows that (a) there will be no one-size-fits-all model/agent. there will continue to be value in specialization (b) the (foundational) model layer will not dominate everything. there will remain headroom in many non-core capabilities, and opportunities for specialized post-training, data/learning signal, harness/context/system-level optimization, etc.

proud to have played a role in making this model good