ABSTRACT An emerging goal in neuroscience is tracking what information is represented in brain activity over time as a participant completes some task. Whilst EEG and MEG offer millisecond temporal resolution of how activity patterns emerge and evolve, standard decoding methods present significant barriers to interpretability as they obscure the underlying spatial and temporal activity patterns. We instead propose the use of a generative encoding model framework that simultaneously infers the multivariate spatial patterns of activity and the variable timing at which these patterns emerge on individual trials. An encoding model inversion allows predictions to be made about unseen test data in the same way as in standard decoding methodology. These SpatioTemporally Resolved MVPA (STRM) models can be flexibly applied to a wide variety of experimental paradigms, including classification and regression tasks. We show that these models provide insightful maps of the activity driving predictive accuracy metrics; demonstrate behaviourally meaningful variation in the timing of pattern emergence on individual trials; and achieve predictive accuracies that are either equivalent or surpass those achieved by more widely used methods. This provides a new avenue for investigating the brain’s representational dynamics and could ultimately support more flexible experimental designs in future. HIGHLIGHTS We introduce SpatioTemporally Resolved MVPA (STRM), an approach that explicitly models how successive stages of stimulus processing are distributed in both space and time in M/EEG data. We show that STRM is broadly applicable to diverse types of M/EEG data and outputs meaningful and interpretable maps of how neural representations evolve in space and time at millisecond resolution. The trial-specific deviations in activity pattern timings identified by STRM are not random, but vary systematically with inter-trial differences in behavioural, cognitive and physiological variables. These methods result in predictive accuracy metrics that are mostly equivalent to, or a modest improvement on, conventional methods.

Spatiotemporally Resolved Multivariate Pattern Analysis for M/EEG