Abstract Mammalian development is associated with extensive changes in gene expression, chromatin accessibility, and nuclear structure. Here, we follow such changes associated with mouse embryonic stem cell differentiation and X inactivation by integrating, for the first time, allele-specific data obtained by high-throughput single-cell RNA-seq, ATAC-seq, and Hi-C. In differentiated cells, contact decay profiles, which clearly distinguish the active and inactive X chromosomes, reveal loss of the inactive X-specific structure at mitosis followed by a rapid reappearance, suggesting a ‘bookkeeping’ mechanism. In differentiating embryonic stem cells, changes in contact decay profiles are detected in parallel on both the X chromosomes and autosomes, suggesting profound simultaneous reorganization. The onset of the inactive X-specific structure in single cells is notably delayed relative to that of gene silencing, consistent with the idea that chromatin compaction is a late event of X inactivation. Novel computational approaches to effectively align single-cell gene expression, chromatin accessibility, and 3D chromosome structure reveal that long-range structural changes to chromosomes appear as discrete events, unlike progressive changes in gene expression and chromatin accessibility.