i get asked VERY frequently to chat about data / RL envs / new companies coming into the space. all of the questions tend to be exactly the same so i want to summarize my thoughts: Q: "Should I start a data company?" A: probably not, unless you have some sort of alpha or have seen what frontier RL envs are like. basically, if you're not deep within an industry, seen data via a lab or working at another data company, or very well connected with lab researchers, you probably should not. though, if its your passion, now is the best time and it'll get increasingly worse. data is also much harder now than it was 6 months ago, and quality standards are ever-increasing. Q: "Is there a product in the space?" A: i think there are repeatable systems that can be created. i.e. QA systems, faster task generation, infrastructure around rollouts/VMs/GPU utilization/prod. system emulation etc. i think data is inherently bespoke, luckily LLMs allow you to harness bespoke intellect infinitely. i don't think it's a SaaS style product where it serves everyone immediately, but more of a quant-desk type system with alpha on how to do specific tasks embedded within an engineer-system like manner. each eng will have a desk that serves them specifically that unlocks 10x, 100x, 1000x capabilities. Q: "Is {insert set} data valuable" A: depends on what you do with it. raw data dumps are cool but it's a lot more about what you can turn it into. Q: "How does {insert data company} differ from {insert competitor}?" OR "Should I join {insert data company}?" A: if you're in the "big, legacy" data companies with contractors, it doesn't really differ that much. if i had to pick, i'd probably join mercor or surge from the big contractor companies, by virtue of early employees/founders being extremely smart. however, i'd much rather join a small, research-heavy data company. they are by far the companies with the highest slope. Q: "Won't labs just in-house this anyway?" A: (1) data is an additive game that isn't zero-sum. just because labs are increasing synthetic data creation internally does not mean they won't buy more data. especially given it is an arms race to the top of intellect, if you can just make valuable data that moves the needle, it doesn't really matter if they can replicate it, it's not worth their researcher time if you solve a problem. the caveat being you have to actually add something yourself, and not just be a parrot who takes requests. saving researcher time is the name of the game. (2) i think this logic applies to everything, if you adopt this mentality, you probably should not work at any applied AI company, since they'll come after or absorb most products in the space.

genuinely a groundbreaking finding, but it also signals to me exactly how important pre-training & post-training data is. given that recursive generalization is essentially proven: > the learning models do is related, thus bad data will lead to bad learning and create polysemanticity where it potentially isn't needed. this can muddle the quality of the neuron and its firing. moreover, for frontier model labs, this can muddle the interpretability of that neuron, by giving it mixed signals. > classifying neurons are especially interesting here. nick breaks down the different organizational structures (i.e. a Colorado neuron or a health SEO neuron) of neurons. if i was a lab right now, i would be trying to understand emergent organizations in my models neurons to understand what fires during failures in training / evals. then, think about tangentially related data in the same activation that can boost the models performance without training only on in-distribution tasks. i.e. if the neuron generalizes, potentially data that sharpens that neuron could improve performance too. > once more interpretability research is done, if there are emergent patterns in every big model that are cross-shared, start thinking about datasets that can target specific neuron performances seems interesting. not sure if this is useful, but could be a new way of understanding model weaknesses, and seeing if there are pre-training / post-training gaps that can be found via exciting directions in evals.