when data vendors call themselves "neolabs" (fleet) or "research partners" (turing), they usually mean one of three things: 1) marketing larp (90% of people) 2) the vendor has made it past the pilot stage. researchers will consider their data proposals, and the vendor can influence what gets procured. this is not necessarily because the lab thinks they're smarter, but because the vendor knows what other labs are buying 3) the vendor is actually innovating on new data shapes. this starts with teams working directly with oai/ant on non-vanilla task types, helping define what frontier training data should even look like. that work eventually trickles down to n-1 labs. +1 if it's exclusive contracts there are probably <10 serious companies in 3)

tbh after reading up on these guys, they're one of the few data cos that are doing it right. hats off to the team

the best environment engineers at moatless synth data companies make 500k base salaries and EASILY generate tens of millions for the co. they eventually had to have asked themselves, "why don't i just leave and sell to multiple labs independently?" obviously this is true for employees in general (surplus value), but data is different - if you leave decagon and start selling customer support software, good luck with that. if you left a data company once you understood what sellable data looks like, you could’ve just made it yourself and sold it across labs since frontier labs were buying everything. this is no longer the case for many reasons (labs in-housing synth pipelines, closed off to new vendors, etc.) for env engineers who understood this, why did you not leave and make your bread while you still had the chance? not looking for bs or obvious answers like: - to learn more (aka extract intel) - belief in equity (only really makes sense if you were early) - culture/camaraderie (alright bro)

there's just no way this griftslop is the same valuation as Linear at 100M ARR

an overlooked reason the leading data cos won't capture the whole market is they can't retain talent. there's a new person leaving mercor to spin out and sell data independently every week

biggest issue with data vendors: overpromising and underdelivering this is obvious, but many ignore it. you'll pitch a lab a higher volume/capacity than you can actually do, and then you have to decide between shipping not much high quality data (disappointing) or a ton of shit unusable data (even worse) have seen so many companies hardly make it through the pilot, and proceed to fail the scale-up when they do

Edwin Chen’s tweets STAY flopping

if you sell your rl data to brokers, consider yourself no better than a third world mercor expert

still bullish on @Sbhaiwala03 tho

warning: if you’re talking to a lab and give someone a poor dataset or samples, you’ve just blacklisted yourself from them. there is no second chance. and everyone in rl data business talks to each other btw. once you’re in, you’re in. but once you’re out, you’re out.

there are people building post-training datasets who have not ONCE post-trained in their life

it’s so obvious why prime intellect and openenv are losing to harbor: 1. they are crazy open source, through and through (qa + task iteration done fully in git PRs), easy to trust 2. simple af to understand the sdk 3. have the other widely used “staple” benchmark, terminal-bench, besides swe-bench prime intellect: no one gaf about their benches until recently, and don’t have open source beyond the sdk openenv: no known bench or open source beyond sdk. somehow they have all the big vendor logos on their repo but ev still silently using harbor

funnily enough, spoke to another guy at xai who said he doesn’t help companies get contracts but he’ll charge 3k per hour to check out your dataset