The potential association between rare germline genetic variants and prostate cancer (PrCa) susceptibility has been understudied due to challenges with assessing rare variation. Furthermore, although common risk variants for PrCa have shown limited individual effect sizes, their cumulative effect may be of similar magnitude as high penetrance mutations. To identify rare variants associated with PrCa susceptibility, and better characterize the mechanisms and cumulative disease risk associated with common risk variants, we analyzed large population-based cohorts, custom genotyping microarrays, and imputation reference panels in an integrative study of PrCa genetic etiology. In particular, 11,649 men (6,196 PrCa cases, 5,453 controls) of European ancestry from the Kaiser Permanente Research Program on Genes, Environment and Health, ProHealth Study, and California Men's Health Study were genotyped and meta-analyzed with 196,269 European-ancestry male subjects (7,917 PrCa cases, 188,352 controls) from the UK Biobank. Six novel loci were genome-wide significant in our meta-analysis, including two rare variants (minor allele frequency < 0.01, at 3p21.31 and 8p12). Gene-based rare variant tests implicated a previously discovered PrCa gene ( HOXB13 ) as well as a novel candidate ( ILDR1 ) highly expressed in prostate tissue. Haplotypic patterns of long-range linkage disequilibrium were observed for rare genetic variants at HOXB13 and other loci, reflecting their evolutionary history. Furthermore, a polygenic risk score (PRS) of 187 known, largely common PrCa variants was strongly associated with risk in non-Hispanic whites (90th vs. 10th decile OR = 7.66, P = 1.80*10-239). Many of the 187 variants exhibited functional signatures of gene expression regulation or transcription factor binding, including a six-fold difference in log-probability of Androgen Receptor binding at the variant rs2680708 (17q22). Our finding of two novel rare variants associated with PrCa should motivate further consideration of the role of low frequency polymorphisms in PrCa, while the considerable effect of PrCa PRS profiles should prompt discussion of their role in clinical practice.