Prefrontal cortex as a meta-reinforcement learning system nature.com/articles/s41593-0… 1. The traditional dopamine theory: The canonical model proposes that dopamine (DA) signals reward prediction errors (RPEs), strengthening or weakening synaptic connections to reinforce associations between situations, actions, and rewards. 2. The prefrontal cortex (PFC) also supports learning: Evidence shows that the PFC represents action values, reward history, previous choices, and upcoming actions, suggesting that it can implement its own reward-based learning computations. 3. A new theory — meta-reinforcement learning (meta-RL): Rather than treating dopamine and the PFC as separate, competing learning systems, the theory proposes that dopamine trains the PFC network to develop its own independent learning algorithm. 4. The brain as a recurrent neural network: The authors model the PFC, together with interconnected basal ganglia and thalamic regions, as a recurrent neural network that integrates sensory inputs, previous actions, and reward information to guide decisions. 5. Two learning timescales: Dopamine-driven synaptic plasticity operates across extended experience, shaping network connectivity. Once trained, the PFC's neural activity dynamics support faster learning within individual tasks, even when synaptic weights remain fixed. 6. Adaptive learning rates: The model adjusts its learning rate according to environmental volatility. It learns faster when reward probabilities change frequently and more slowly when conditions remain stable, resembling observed human behavior. 7. Explaining reward prediction errors and inferred values: The framework explains how dopamine signals can reflect inferred changes in reward value, even when an outcome has not been directly experienced. The PFC's learned representation of task structure can influence the predictions contributing to dopamine signals. 8. Model-based behavior can emerge from model-free training: Simulations of the two-step decision-making task produced behavior consistent with model-based learning. This suggests that dopamine-driven training can create PFC dynamics capable of exploiting task structure without requiring a separate model-based learning algorithm during testing. 9. Learning to learn: After training across many related tasks, the model rapidly adapts to unfamiliar stimuli and can learn new reward associations from very little experience. This mirrors the improvement in learning efficiency observed in animal experiments. 10. Dopamine may have a dual role: In addition to modifying synaptic plasticity, dopamine signals may convey reward information that directly influences ongoing PFC activity. The framework therefore offers an alternative interpretation of optogenetic experiments that manipulate dopamine and change decision-making behavior. Key takeaway: The central idea is that dopamine teaches the prefrontal cortex how to learn. Dopamine-driven synaptic changes gradually shape a recurrent neural network whose activity can then implement a faster, flexible learning algorithm. By connecting neuroscience with recurrent neural networks and meta-learning in AI, this theory offers a unified computational explanation for reward-based learning, adaptive decision-making, and learning to learn.