Aim Vitamin D (VD) affects skeletal muscles. The high prevalence of VD deficiency in Japan may lead to decreased skeletal muscle mass and strength, increasing the prevalence of sarcopenia. Therefore, we aimed to investigate the association between serum VD levels and skeletal muscle indices in a Japanese community‐dwelling older population. Methods We extracted data from the Septuagenarians, Octogenarians, Nonagenarians Investigation with Centenarians (SONIC) study. We analyzed the data for participants in the 70s and 90s age groups. Skeletal mass index (SMI) using bioimpedance analysis, grip strength, walking speed, and serum VD levels using 25‐hydroxyvitamin D [25(OH)D] were measured. Results We analyzed the data of 310 participants in their 70s and 48 in their 90s. Mean serum 25(OH)D levels were 21.6 ± 5.0 ng/mL in the 70s group and 23.4 ± 9.1 ng/mL in the 90s group. In the 70s group, serum 25(OH)D levels correlated with SMI ( r = 0.21, P < 0.0001) and grip strength ( r = 0.30, P < 0.0001). Serum 25(OH)D levels were independently associated with SMI after adjusting for sex, body mass index, and serum albumin levels. In the 90s group, serum 25(OH)D levels were correlated with SMI ( r = 0.29, P = 0.049) and grip strength ( r = 0.34, P = 0.018). However, the multivariate analysis showed no independent association between SMI, grip strength, and serum 25(OH)D levels. Conclusion In a cross‐sectional analysis of an older population, serum VD levels were associated with SMI and grip strength, and this association was more pronounced in the 70s group than in the 90s group. Our results suggest that serum VD levels maintain skeletal muscle mass and grip strength. Geriatr Gerontol Int 2024; ••: ••–•• .