Gene-based tests to study the combined effect of rare variants towards a particular phenotype have been widely developed for case-control studies, but their evolution and adaptation for family-based studies, especially for complex incomplete families, has been slower. In this study, we have performed a practical examination of all the latest gene-based methods available for family-based study designs using both simulated and real datasets. We have examined the performance of several collapsing, variance-component and transmission disequilibrium tests across eight different software and twenty-two models utilizing a cohort of 285 families (N=1,235) with late-onset Alzheimer disease (LOAD). After a thorough examination of each of these tests, we propose a methodological approach to identify, with high confidence, genes associated with the studied phenotype with high confidence and we provide recommendations to select the best software and model for family-based gene-based analyses. Additionally, in our dataset, we identified PTK2B, a GWAS candidate gene for sporadic AD, along with six novel genes (CHRD, CLCN2, HDLBP, CPAMD8, NLRP9, MAS1L) as candidates genes for familial LOAD.

Evaluation of gene-based family-based methods to detect novel genes associated with familial late onset Alzheimer disease