Unravelling how gene regulatory networks are remodelled during evolution is crucial to understand how species adapt to environmental changes. We addressed this question for X-chromosome inactivation, a process essential to female development that is governed, in eutherians, by the XIST lncRNA and its cis-regulators. To reach high resolution, we studied closely related primate species, spanning 55 million years of evolution. We show that the XIST regulatory circuitry has diversified extensively over such an evolutionary timeframe. The insertion of a HERVK transposon has reshuffled XIST 3D interaction network in macaque embryonic stem cells (ESC) and XIST expression is maintained by the additive effects of the JPX lncRNA gene and a macaque specific enhancer. In contrast, JPX is the main contributor to XIST expression in human ESCs but is not significantly involved in XIST regulation in marmoset ESCs. None of these entities are however under purifying selection, which suggests that neutrally evolving non-coding elements harbour high adaptive potentials.