Abstract Machine learning derived embeddings are a compressed representation of high content data modalities obtained through deep learning models[1]. Embeddings have been hypothesized to capture detailed information about disease states and have been qualitatively shown to be useful in genetic discovery. Despite their promise, embeddings have some drawbacks: i) they are often confounded by covariates, and ii) their disease relevance is hard to ascertain. In this work we describe a framework to systematically evaluate the utility of embeddings in genetic discovery called EmbedGEM ( Embed ding G enetic E valuation M ethods). Although, motivated by applications to embeddings, EmbedGEM is equally applicable for other multivariate traits as well. EmbedGEM focuses on comparing embeddings along two axes: i) heritability of the embeddings, and ii) ability to identify ‘disease relevant’ variants. We use the number of genome-wide significant signals and mean/median chi-square statistic as a proxy for the heritability of multivariate traits. To evaluate disease relevance, we compute polygenic risk scores for each orthogonalized component of the embedding (or multivariate comparators) and evaluate their association with a held-out set of patients with high-confidence disease traits. While we introduce some relatively straightforward ways to evaluate heritability and disease relevance, we foresee that our framework can be easily extended by adding more metrics. We demonstrate the utility of EmbedGEM by using it to evaluate embedding and non-embedding traits in two separate datasets: i) a synthetic dataset simulated to demonstrate the ability of the framework to correctly rank traits based on their heritability and disease relevance, ii) data from the UK Biobank focused on NAFLD relevant traits. EmbedGEM is implemented in the form of an easy to use Python-based workflow ( https://github.com/insitro/EmbedGEM ).

EmbedGEM: A framework to evaluate the utility of embeddings for genetic discovery