[3/] Some significant results are: 1. We initially tried refining over 30 passes, but found that vocabulary and vocabulary size largely settled by the 8th pass, a phenomenon we term schematic convergence; 2. The User Models produced indicate significant transition structures (they can predict what the user will do next, and why with ~6pp greater accuracy and ~4pp lower perplexity than the best naive baselines). 3. Traditionally, the field prefers closed taxonomies; Having your LLM freely annotate leads to vocabulary explosion, and usually makes comparison/modelling difficult. The tradeoff is that constraining vocabulary means your model completely fails to label states you didn’t anticipate. We solve this by having open-ended label generation, while preventing vocabulary explosion by having sliding-window context at annotation time, and normalization to further collapse semantically-near labels. 4. Human annotators largely agreed (krippendorf’s alpha = 0.4) that SERUM’s iterative refinement produced more useful/accurate labels ~80% of the time compared to a naive first pass labelling.

day2 of the fsd ai game development test The game dev agent (opus 5) made a starvation mechanic; one of the machiavellian personas built trust with the other 3 players by never getting involved with price gouging, cartelling or raids… but while the other 3 were fighting and desperate for resources, it bought up all the food and starved everyone 💀

1/ I’ve started an ai game dev company test; - claude code opus 4.8 [1M] on max plan with ultracode, - /loop with “keep going!… refactor code… assess playabillity…” - essentially, can it balance an economy strategy game & make it fun (game theory principles)