Abnormal development and function of the hippocampus are two of the most consistent findings in humans and rodents exposed to early life adversity, with males often being more affected than females. Using the limited bedding (LB) paradigm as a rodent model of early life adversity, we found that male adolescent mice that had been exposed to LB exhibit significant deficits in contextual fear conditioning and synaptic connectivity in the hippocampus, which are not observed in females. This is linked to altered developmental refinement of connectivity, with LB severely impairing microglial-mediated synaptic pruning in the hippocampus of male and female pups on postnatal day 17 (P17), but not in adolescent P33 mice when levels of synaptic engulfment by microglia are substantially lower. Since the hippocampus undergoes intense synaptic pruning during the second and third weeks of life, we investigated whether microglia are required for the synaptic and behavioral aberrations observed in adolescent LB mice. Indeed, transient ablation of microglia from P13-21, in normally developing mice caused sex-specific behavioral and synaptic abnormalities similar to those observed in adolescent LB mice. Furthermore, chemogenetic activation of microglia during the same period reversed the microglial-mediated phagocytic deficits at P17 and restored normal contextual fear conditioning and synaptic connectivity in adolescent LB male mice. Our data support an additional contribution of astrocytes in the sex-specific effects of LB, with increased expression of the membrane receptor MEGF10 and enhanced synaptic engulfment in hippocampal astrocytes of 17-day-old LB females, but not in LB male littermates. This finding suggests a potential compensatory mechanism that may explain the relative resilience of LB females. Collectively, these studies highlight a novel role for glial cells in mediating sex-specific hippocampal deficits in a mouse model of early-life adversity.