Quantifying the genetic correlation between cancers can provide important insights into the mechanisms driving cancer etiology. Using genome-wide association study summary statistics across six cancer types based on a total of 296,215 cases and 301,319 controls of European ancestry, we estimate the pair-wise genetic correlations between breast, colorectal, head/neck, lung, ovary and prostate cancer, and between cancers and 38 other diseases. We observed statistically significant genetic correlations between lung and head/neck cancer (rg=0.57, p=4.6x10-8), breast and ovarian cancer (rg=0.24, p=7x10-5), breast and lung cancer (rg=0.18, p=1.5x10-6) and breast and colorectal cancer (rg=0.15, p=1.1x10-4). We also found that multiple cancers are genetically correlated with non-cancer traits including smoking, psychiatric diseases and metabolic characteristics. Functional enrichment analysis revealed a significant excess contribution of conserved and regulatory regions to cancer heritability. Our comprehensive analysis of cross-cancer heritability suggests that solid tumors arising across tissues share in part a common germline genetic basis.