Influenced by factors such as brain states and behavior, neurons exhibit substantial response variability even to identical stimuli. Because these factors are non-stationary, they dynamically impact the fidelity of sensory processing. However, it remains unclear how their relative impact on neuronal variability evolves over time. To address this question, we designed an encoding model with latent states to partition visual cortical variability across three crucial categories of sources: internal brain dynamics, behavior, and external visual stimulus. Applying a hidden Markov model to the rhythmic patterns of cortical local field potentials, we consistently identified three distinct oscillation states. Each state revealed a unique variability profile and a consistent descending trend of stimulus modulation across the visual hierarchy. Regression models within each state revealed a dynamic composition of factors contributing to the observed spiking variability, with the primary influencing factor switching within seconds. In the state dominated by high-frequency oscillations, sensory inputs and behavior exerted the most influence on population dynamics. Conversely, internal brain activity explained most of the variance in the state dominated by low-frequency oscillations. This heterogeneity across states underscores the importance of partitioning variability over time, particularly when considering the dynamic influence of non-stationary factors on sensory processing.

Deciphering neuronal variability across states reveals dynamic sensory encoding