Purpose: Despite similarities in upfront treatment of childhood cancer, not every adult survivor of childhood cancer (CCS) has an impaired bone mineral density (BMD). No data are available on the role of genetic variation on impairment of BMD in CCS. Methods: This cross-sectional single-center cohort study included 334 adult CCSs (median follow-up time after cessation of treatment: 15 years; median age at follow-up: 26 years). Total body BMD (BMDTB) and lumbar spine BMD (BMDLS) were measured by dual x-ray absorptiometry. We selected 12 candidate single-nucleotide polymorphisms (SNPs) in 11 genes (COL1A1, TNFSF11, TNFRSF11, TNRFSA11B, VDR, ESR1, WLS, LRP5, MTHFR, MTRR, IL-6). Results: Multivariate analyses revealed that lower BMD was associated with lower weight and height at follow-up, male sex, and previously administered radiotherapy. Survivors with the homozygous minor allele (GG) genotype of rs2504063 (ESR1: estrogen receptor type 1) had a lower BMDTB values (–1.16 vs. –0.82; P = 0.01) than those with the AG/AA genotype; however, BMDLS was not different. Carriers of two minor alleles (GG) of rs599083 (LRP5: low-density lipoprotein receptor) revealed lower BMDTB (–1.20 vs. –0.78; P = 0.02) and lower BMDLS (–0.95 vs. –0.46; P = 0.01) values than those with the TT/TG genotype. Conclusion: CCSs who are carriers of candidate SNPs in the ESR1 or LRP5 genes seem to have an impaired bone mass at an early adult age. Information on genetic variation, in addition to patient- and treatment-related factors, may be helpful in identifying survivors who are at risk for low bone density after childhood cancer treatment.