Modulatory ( e.g. dopaminergic) neurons provide 'teaching signals' that drive associative learning across the animal kingdom, but the circuit mechanisms by which these signals are computed are still unclear. To provide a basis for understanding the circuit implementation of learning algorithms, we generated a synaptic-resolution connectivity map of the circuits upstream of all modulatory neurons in an associative learning center, the mushroom body (MB) of the Drosophila larva. We discovered afferent pathways from sensory neurons and a large number of one-step and two-step feedback pathways originating from MB output neurons. We also found a surprising density of cross-compartment feedback pathways that link distinct memory systems ( e.g. aversive and appetitive). This architecture suggests that the MB functions as an interconnected ensemble during learning and that any previously formed memories of a stimulus can potentially regulate future learning about that stimulus. We functionally confirmed some of the structural pathways and found that some modulatory neurons compare inhibitory input from their own compartment and excitatory input from compartments of opposite valence, potentially enabling them to more accurately compute predicted values of stimuli. We developed a model of the circuit constrained by the connectome and functional data and used it to explore the computational advantages offered by the newly discovered feedback motifs. The model shows that the observed feedback pathways increase the network′s performance on complex learning tasks. It also shows that cross-compartment connections support the computation of predicted values and improve performance on higher-order learning tasks. Our study provides the most detailed view to date of a brain circuit that computes teaching signals and provides insights into the architectural motifs that support reinforcement learning in a biological system.

Multilevel feedback architecture for adaptive regulation of learning in the insect brain