Abstract The ongoing activity of neuronal populations represents an internal brain state that influences how sensory information is processed to control behaviour. Conversely, external sensory inputs perturb network dynamics, resulting in lasting effects that persist beyond the duration of the stimulus. However, the relationship between these dynamics and circuit architecture and their impact on sensory processing, cognition and behaviour are poorly understood. By combining cellular-resolution calcium imaging with mechanistic network modelling, we aimed to infer the spatial and temporal network interactions in the zebrafish optic tectum that shape its ongoing activity and state-dependent responses to visual input. We showed that a simple recurrent network architecture, wherein tectal dynamics are dominated by fast, short range, excitation countered by long-lasting, activity-dependent suppression, was sufficient to explain multiple facets of population activity including intermittent bursting, trial-to-trial sensory response variability and spatially-selective response adaptation. Moreover, these dynamics also predicted behavioural trends such as selective habituation of visually evoked prey-catching responses. Overall, we demonstrate that a mechanistic circuit model, built upon a uniform recurrent connectivity motif, can estimate the incidental state of a dynamic neural network and account for experience-dependent effects on sensory encoding and visually guided behaviour.