Low skeletal muscle mass (LSMM) is increasingly recognized for its predictive value for adverse events in cancer patients. In specific, the predictive value of LSMM has been demonstrated for anti-cancer drug toxicity in a variety of cancer types and anti-cancer drugs. However, due to the limited sample size and study populations focused on a single cancer type, an overall predictive value of LSMM for anti-cancer drug toxicity remains unknown. Therefore, this review aims to provide a comprehensive overview of the predictive value of LSMM and perform a meta-analysis to analyse the overall effect. A systematic search was conducted of MEDLINE, Scopus, EMBASE, and Cochrane. Inclusion criteria were skeletal muscle mass (SMM) evaluated with computed tomography (CT) or magnetic resonance imaging (MRI), articles published in English, SMM studied in humans, SMM measurement normalized for height, and patients did not receive an intervention to treat or prevent LSMM. A meta-analysis was performed using a random-effects model and expressed in odds ratio (OR) with 95% confidence interval (CI). Heterogeneity was assessed using χ2 and I2 statistics. The search yielded 907 studies. 31 studies were included in the systematic review. Sample sizes ranged from 21 to 414 patients. The occurrence of LSMM ranged from 12.2% to 89.0%. The most frequently studied cancer types were oesophageal, renal, colorectal, breast, and head and neck cancer. Patients with LSMM had a higher risk of severe toxicity (OR 4.08; 95% CI 2.48–6.70; p < 0.001) and dose-limiting toxicity (OR 2.24; 95% CI 1.28–3.92; p < 0.001) compared to patients without LSMM. To conclude, the predictive value of LSMM for anti-cancer drug toxicity can be observed across cancer types. This information increases the need for further research into interventions that could treat LSMM as well as the possibility to adapt treatment regimens based on the presence of LSMM.