The animal model is a key tool in quantitative genetics and has been used extensively to estimate fundamental parameters, such as additive genetic variance, heritability, or inbreeding effects. An implicit assumption of animal models is that all founder individuals derive from a single population. This assumption is commonly violated, for instance in crossbred livestock breeds, when an observed populations receives immigrants, or when a meta-population is split into genetically differentiated subpopulations. Ignoring genetic differences among different source populations of founders may lead to biased parameter estimates, in particular for the additive genetic variance. To avoid such biases, genetic group models, extensions to the animal model that account for the presence of more than one genetic group, have been proposed. As a key limitation, the method to date only allows that the breeding values differ in their means, but not in their variances among the groups. Methodology previously proposed to account for group-specific variances included terms for segregation variance, which rendered the models infeasibly complex for application to most real study systems. Here we explain why segregation variances are often negligible when analyzing the complex polygenic traits that are frequently the focus of evolutionary ecologists and animal breeders. Based on this we suggest an extension of the animal model that permits estimation of group-specific additive genetic variances. This is achieved by employing group-specific relatedness matrices for the breeding value components attributable to different genetic groups. We derive these matrices by decomposing the full relatedness matrix via the generalized Cholesky decomposition, and by scaling the respective matrix components for each group. To this end, we propose a computationally convenient approximation for the matrix component that encodes for the Mendelian sampling variance. Although convenient, this approximation is not critical. Simulations and an example from an insular meta-population of house sparrows in Norway with three genetic groups illustrate that the method is successful in estimating group-specific additive genetic variances and that segregation variances are indeed negligible in the empirical example. Quantifying differences in additive genetic variance within and among populations is of major biological interest in ecology, evolution, and animal and plant breeding. The proposed method allows to estimate such differences for subpopulations that form a connected meta-population, which may also be useful to study temporal or spatial variation of additive genetic variance.

Animal models with group-specific additive genetic variances: extending genetic group models