building data programs for frontier labs · evals · rl envs Head of AI Engagements @pareto_ai CS @DavidsonCollege

Charlotte, NC
Joined July 2024
Dimitrios Chavouzis retweeted
human data still tiptoes around the s-word, but a lot of what’s sold as “human” is already part synthetic, from humans reviewing model samples to models expanding human seeds. the industry leans heavily on model participation as the pool of experts who can author tasks hard enough to break frontier models is outpaced by demand for ever more realistic, complex data. in our internal ablations, human-seeded augmented tasks were statistically indistinguishable from fully human-authored ones on both domain-specific knowledge and agentic tool-use benchmarks. provenance is still treated as a proxy for quality and realism. i expect the two to decouple as synthetic data methods mature and the industry adopts more sophisticated measures of quality.
4
1
42
3,213