Mammalian stem-cell-based models of embryo development (stembryos) hold great promise in basic and applied research. However, considerable phenotypic variation despite identical culture conditions limits their potential. The biological processes underlying this seemingly stochastic variation are poorly understood. Here, we investigate the roots of this phenotypic variation by intersecting transcriptomic states and morphological history of individual stembryos across stages modeling post-implantation and early organogenesis. Through machine learning and integration of time-resolved single-cell RNA-sequencing with imaging-based quantitative phenotypic profiling, we identify early features predictive of the phenotypic end-state. Leveraging this predictive power revealed that early imbalance of oxidative phosphorylation and glycolysis results in aberrant morphology and a neural lineage bias that can be corrected by metabolic interventions. Collectively, our work establishes divergent metabolic states as drivers of phenotypic variation, and offers a broadly applicable framework to chart and predict phenotypic variation in organoid systems. The strategy can be leveraged to identify and control underlying biological processes, ultimately increasing the reproducibility of in vitro systems.