Happy to introduce DataOrchestra by @Z_Huang_02 and GAIR NLP, and feel thrilled to participate!! Most pretraining pipelines ask: Which processing recipe should we apply to the corpus? DataOrchestra asks a more counterintuitive question: What should happen to this one example? Instead of forcing every sample through the same fixed pipeline, it orchestrates the curation process of every pretraining example. 🎼 DataOrchestra is an open-source framework that turns data curation into a per-example decision problem. Given a raw chunk, a lightweight 1.7B orchestrator decides whether to: raw chunk → Drop → Untouch → Clean For chunks that need cleaning, it selects only the necessary operations—Noise Pruning, Surface Rectification, and Pedagogical Augmentation—and generates a concrete instruction for every selected rewriting stage. A plausible processing plan may still fail when executed by the actual tool models. DataOrchestra therefore learns from execution feedback: teacher plan → tool execution → verification → plan evolution → orchestrator training 🔧 This process produces 300K tool-grounded training pairs. Models pretrained on DataOrchestra-curated data consistently outperform fixed and individual processing strategies across 11 benchmarks, from 0.5B to 7B scale. The advantage grows with model size. Per-example routing also avoids unnecessary work: around 35% of chunks stop at Drop or Untouch, reducing curation compute by approximately 37% compared with always running the full NP → SR → PA pipeline. The takeaway is simple: Do not process pretraining data uniformly. Orchestrate every example precisely. Code, models, prompts, and the complete pipeline ↓