Abstract Alzheimer’s disease and related dementias (ADRD) are marked by intracellular tau aggregates in the medial-temporal lobe (MTL) and extracellular amyloid aggregates in the default network (DN). Here, we sought to clarify ADRD-related co-dependencies between the MTL’s most vulnerable structure, the hippocampus (HC), and the highly associative DN at a subregion resolution. We confronted the effects of APOE ɛ2 and ɛ4, rarely investigated together, with their impact on HC-DN co-variation regimes at the population level. In a two-pronged decomposition of structural brain scans from ∼40,000 UK Biobank participants, we located co-deviating structural patterns in HC and DN subregions as a function of ADRD family risk. Across the disclosed HC-DN signatures, recurrent deviations in the CA1, CA2/3, molecular layer, fornix’s fimbria, and their cortical partners related to ADRD risk. Phenome-wide profiling of HC-DN co- variation expressions from these population signatures revealed male-specific associations with air-pollution, and female-specific associations with cardiovascular traits. We highlighted three main factors associated with brain- APOE associations across the different gene variants: happiness, and satisfaction with friendships, and with family. We further showed that APOE ɛ2/2 interacts preferentially with HC-DN co-variation patterns in estimating social lifestyle in males and physical activity in females. Our findings reinvigorate the often-neglected interplay between APOE ɛ2 dosage and sex, which we have linked to fine-grained structural divergences indicative of ADRD susceptibility.