The fact that this assigns useful credit 1000 layers deep when such works typically managed a couple dozen layers is quite a feat. Congrats to @SakanaAILabs and the authors @jeffreyseely & Julian Gould! I do think one distinction is important though. PC-ALM and related approaches are not very different from backpropagation in the strongest sense. It is a dynamical mechanism for recovering a backprop-like credit signal. The network is still trying to communicate the same global supervised error through the hierarchy. The change is very worthwhile to study, is indeed more bio-plausible than backpropagation and could have some computational advantages for asynchronous updates, but it also means adding machinery to approximate the signal that backprop already computes directly. It also means that the ceiling in computational efficiency, sample efficiency and generalization is likely that of backprop. On our side, we have been working for a long time on a more radical route: remove the backward credit signal altogether. In our approach, learning is purely bottom-up. There is no feedback signal, no target propagated through the network, and no global task loss dictates hidden weight updates. The causal arrow is reversed: local weight changes give rise to learning at the level of the whole model. In what we showed publicly several years ago, such feedback-free approaches can score better than PC-ALM and its predecessors report, despite being arguably more biologically plausible, more computationally efficient, and more suitable for hardware implementations. In the multilayer SoftHebb work from 2022, in the full 1000-class ImageNet problem – not Tiny ImageNet – and other tasks, we showed accuracies that PC-ALM and other recent approaches have not managed to reach. Interestingly, these results were reached after a single presentation (i.e. epoch) of 10% of the dataset. Under different architectures, setups, etc, but the broader point is important. In light of all the recent interest around continual learning, test-time compute, and post-training, here is another interesting aspect: These genuinely local learning rules are also inherently amenable to online, continual, and distributed learning, because they only have a forward pass, they learn without batches, and they have minimal communication requirements across parts of the model. The difference becomes especially interesting for post-GPU hardware architecture. If local dynamics such as PC-ALM's are cheaper than backprop on a physical substrate, then algorithms that do not need the error-propagation dynamics in the first place have an even larger advantage. The difficulty is that once you move far enough away from backprop, you also move away from the conceptual framework that most of modern AI is built and trained around. There is less machinery, but also fewer familiar tools for understanding what the system is doing and how to scale it. That has made this direction harder for others in both academia and industry to follow. Those public results are several years old now, but we at @noemon_ai have kept pushing this direction. We can’t wait to show where this path has led us.

The first valid submission to break 90% on ARC-AGI-2 for a low cost is now on the community leaderboard. @noemon_ai achieved it back in March, and it took four months for the rest of the frontier to catch up, with GPT-5.6 and Claude 5. Thank you, @arcprize @mikeknoop @fchollet @GregKamradt for getting to our pull request! Our request for verification on the semi-private set has also been pending. BTW, we have open-sourced this outer learning harness for the community, in a version that uses simple ICL as the inner learning method, but at @noemon_ai we work on the fundamentals of neural learning.

Many understand this would be huge, but most current solutions are "let's write a summary" and call it a day. Backprop based training doesn't cut it though. Our founding team @noemon_ai has been building the alternative path for years, and we now know how to scale it.