Abstract

There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second-level 3D features, rather than the original 4D fMRI data. This trade-off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called \"parallel group ICA+ICA\" that incorporates temporal fMRI information from GICA into a parallel ICA framework, aiming to enable direct fusion of first-level fMRI features with other modalities (e.g. structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate inter-modality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI-sMRI components that show group difference between schizophrenia and controls in both modalities. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.