RL training for LLMs involves exposure to problems in the “Goldilocks zone” of difficulty: not too hard, not too easy. But difficulty is not the only thing that matters. Problem type matters, too. And LLMs do not see or organize problem types the way humans do. This is the starting point of Manifold Bandits. The fundamental issue is this: when using RL to train LLMs, there will always be problems that are more productive for the policy at a given training iteration. Across a dataset, there are an astronomical number of possible batch orderings; uniform sampling is unlikely to consistently land on the most useful ones by chance. This is why many adaptive curriculum learning and difficulty-aware methods focus on the policy’s “optimal learning zone,” avoiding examples that are already trivial or still impossible for the LLM to solve. But problem type is usually treated differently. Task decompositions within a training set are often manually defined according to human semantics, or ignored entirely, with an entire dataset/domain treated as “one task,” despite the immense heterogeneity seen even in small training datasets. It is generally understood that LLMs may not share our concept of difficulty. But it seems less recognized that LLMs may not share our concept of problem type, either. In both cases, training the LLM according to our own semantics can obscure fine-grained learning dynamics that could otherwise be exploited. In this work, we seek to derive an adaptive curriculum learning method that caters not only to the policy’s ability, but also its perception: its latent organization of tasks. Driven by the manifold hypothesis and the Bitter Lesson, we leverage computation to derive curriculum structure from the policy’s latent geometry, then use Bayesian inference over that structure to guide search and learning over the training set. The result is Bayesian Manifold Curriculum (BMC): an algorithm that does not just try to find problems of the right difficulty, but instead orchestrates training effort across diverse and interacting problem types. More technically, we frame problem sampling for LLMs as a manifold-structured bandit problem with endogenous non-stationarity. The tl;dr is that the problems/arms are diverse, structured, and interact through policy updates, so the typical non-stationary bandit framing is not quite the right fit for training LLMs. BMC is derived from this new framing. 🌐Website: darrienmckenzie.com/manifold… 📰Paper: arxiv.org/abs/2606.19750 💻Code: github.com/DarrienMcKenzie/m… 🧵1/N