Ribonucleoprotein (RNP) granules are membraneless electron-dense structures rich in RNAs and proteins, and involved in various cellular processes. Two RNP granules in male germ cells, intermitochondrial cement and the chromatoid body (CB), are associated with PIWI-interacting RNAs (piRNAs) and are required for transposon silencing and spermatogenesis. Other RNP granules in male germ cells, the reticulated body and CB remnants, are also essential for spermiogenesis. In this study, we disrupted FBXO24, a testis-enriched F-box protein, in mice and found numerous membraneless electron-dense granules accumulated in sperm flagella. Fbxo24 knockout (KO) mice exhibited malformed flagellar structures, impaired sperm motility, and male infertility, likely due to the accumulation of abnormal granules. The amount and localization of known RNP granule-related proteins were not disrupted in Fbxo24 KO mice, suggesting that the accumulated granules were distinct from known RNP granules. Further studies revealed that RNAs and two importins, IPO5 and KPNB1, abnormally accumulated in Fbxo24 KO spermatozoa. In addition, IPO5 and KPNB1 were recruited to stress granules, RNP complexes, when cells were treated with oxidative stress or a proteasome inhibitor. These results suggest that FBXO24 plays a critical role in preventing the accumulation of importins and RNP granules in sperm flagella.