Hypothalamic inflammation underlies diet-induced obesity and diabetes in rodent models. While diet normalization largely allows for recovery from metabolic impairment, it remains unknown whether long-term hypothalamic inflammation induced by obesogenic diets is a reversible process. In this study, we aimed at determining sex specificity of hypothalamic neuroinflammation and gliosis in mice fed a fat- and sugar-rich diet, and their reversibility upon diet normalization. Mice were fed a 60%-fat diet complemented by a 20% sucrose drink (HFHSD) for 3 days or 24 weeks, followed by a third group that had their diet normalized for the last 8 weeks of the study (reverse diet group, RevD). We determined the expression of pro- and anti-inflammatory cytokines, and of the inflammatory cell markers IBA1, CD68, GFAP and EMR1 in the hypothalamus, and analyzed morphology of microglia (IBA-1+ cells) and astrocytes (GFAP+ cells) in the arcuate nucleus. After 3 days of HFHSD feeding, male mice showed over-expression of IL-13, IL-18, IFN-γ, CD68 and EMR1 and reduced expression of IL-10, while females showed increased IL-6 and IBA1 and reduced IL-13, compared to controls. After 24 weeks of HFHSD exposure, male mice showed a general depression in the expression of cytokines, with prominent reduction of TNF-α, IL-6 and IL-13, but increased TGF-β, while female mice showed over-expression of IFN-γ and IL-18. Furthermore, both female and male mice showed some degree of gliosis after HFHSD feeding for 24 weeks. In mice of both sexes, diet normalization after prolonged HFHSD feeding resulted in partial neuroinflammation recovery in the hypothalamus, but gliosis was only recovered in females. In sum, HFHSD-fed mice display sex-specific inflammatory processes in the hypothalamus that are not fully reversible after diet normalization.