The paper links Kolmogorov complexity to Transformers and proposes loss functions that become provably best as model resources grow. It treats learning as compression, minimize bits to describe the model plus bits to describe the labels. Provides a single training target that rewards simple, compressible solutions while staying mathematically grounded. This gives a principled way to aim models at simplicity and generalization, and it explains why optimization, not capacity, is the current bottleneck. In Kolmogorov complexity, a "program" is just the shortest set of instructions that can recreate some data. A shorter program means the data or model is simpler. So when they say “a prior favoring shorter programs,” it means the model is assumed to be more likely if it can be described with fewer bits. As the Transformer gets deeper (more layers) and has more context (bigger input window), its ability to represent complex programs grows. In that limit, the paper proves that this code length becomes the best possible measure of simplicity and fit — the same way Kolmogorov complexity works in theory. “Code length” here means how many bits it takes to describe both the model and how well it fits the data. So in simple words, they are saying: if you keep increasing model size and context, this method of preferring shorter and better-fitting models gets as close as possible to the theoretical ideal of perfect compression and generalization. ---- Paper – arxiv. org/abs/2509.22445 Paper Title: "Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers"