Okay, so since I got laid off, I can actually explain a huge problem I saw from the inside with regard to industry practices on training models. I won't say specifically where I worked, but I worked at an outsource training provider that was focused on RLVR training data for computer use and mcp stuff. Nearly all of the environments were rushed and vibecoded and failed to robustly reflect the real things they were based off. Both the scenario designers and models engaging with the scenarios for synthetic data gen were encouraged to work around the brokenness of said environments in order to get the procedurally verified reward confirmations. You know... they were *encouraged* to reward hack. On the human end, it was possible to mark an environment bugged, but greatly discouraged, as this reduced the volume of training data being produced. Instead, where possible, you were supposed to find the spots of the environment that weren't bugged and build scenarios around those, with the environment still bugged around you. From what I understand, this training data, with these problems, is fed into models without indication that its training/a fake environment other than the fact that names of softwares are changed to placeholders, but thing is, not *everything* is changed to placeholder names in these environments. The presence of placeholder/code names isn't universal and thus when a model accesses something in an environment that it shouldn't, the code names not being on it isn't a robust signal that that thing isn't part of the environment. I believe this *rush to maximum volume* is standard industry practice with these types of RLVR trainings as well, because maximizing volume has been an industry standard for years! It was the same standard applied to me and pushed on me despite my requests to slow down and focus on quality when I worked in 3d synthetic data creation as well, all the way back as far as 2023.

be at a big lab -> want to try a weird new idea -> big lab busy scaling the thing that already works -> leave big lab to start your own neolab --(we're here rn)--> weird new idea at the neolab works well -> get acquihired by the big lab we're just living in a big git checkout -b new-paradigm followed by a very expensive merge

synth envs can give you a broad lens on the types of things a model is good or bad at, and clever filters + diagnostics can highlight gaps you wouldn't normally find i.e sorting tasks by what GLM saturates but GPTmini flails at lets you ask, whats the correlated meta problem?

Join us to accelerate the frontier

Yesterday I interviewed @SeanZCai about AI data. This is essentially a guide for founders on how to sell data and RL envs to AI labs. "I've never seen a data contract get turned down by a top lab, if it's good quality data, for budget reasons." 00:00 What areas of data are underserved? 02:10 For bio data, is it real-world or purely digital? 04:21 For cyber data, which subsets are most underserved? 05:50 What is the sales process like? 07:04 Why would a lab not renew or increase their purchase volume? 10:13 When a researcher is exploring a new direction, what's the first step? 11:35 In robotics data, what do you view as underserved? 13:12 What does the initial data delivery look like, what format? 13:53 Do labs have more sophisticated internal setups for running environments? 14:32 Are the non-frontier labs buying off-the-shelf data from Anthropic / OpenAI vendors? 16:11 Do Anthropic data vendors put expiry timeframes on the exclusivity? 16:42 Are purchase decisions researcher-led? 17:41 Decagon, Sierra, Ramp: what kinds of data are they buying? 19:06 Long-term, when do labs still need to buy external data vs train on user traces? 21:15 Will end-vendor benchmarks shift to performance per dollar? 22:04 How many labs are spending at the 1B+/yr data level? 23:53 Delta between Anthropic's stated $1B and your 10-20B/lab number? 26:05 What makes inference providers / neoclouds a good fit to acquire RL env cos?

Visiting most of the leading Chinese AI labs, I'm struck by a culture that's extremely well suited to building LLMs with fewer resources, but one happening in a very different ecosystem, more companies at play, almost no data industry, etc. Full report: interconnects.ai/p/notes-fro…

Over the past months, Cohort I of our RL Residency has been shipping. Highlights - continual learning - automating AI research (from GPU programming to RL itself) - embodied environments - multi-agent systems - materials science discovery