Musicians are often regarded as a positive example of brain plasticity and associated cognitive benefits. This emerges when experienced musicians (e.g., musicians with over ten years of music training and practice) are compared with nonmusicians. A frequently observed behavioral finding is a short-term memory advantage of the former over the latter. Although available meta-analysis reports that the effect size of this advantage is medium (Hedges’ g = 0.5), no literature study was adequately powered to estimate reliably an effect of such size. This multi-lab study has been ideated, realised, and conducted in-lab by several groups that have been working on this topic. Our ultimate goal was to provide a community-driven shared and reliable estimate of the musicians’ short-term memory advantage (if any) and set a method and a standard for future studies in neuroscience and psychology comparing musicians and nonmusicians. Thirty-three research units recruited a total of 600 experienced musicians and 600 nonmusicians, a number that is sufficiently large to estimate a small effect size (Hedges’ g = 0.3) with a high statistical power (i.e., 95%). Subsequently, we measured the difference in short-term memory for musical, verbal, and visuospatial stimuli. We also looked at cognitive, personality, and socioeconomic factors that might mediate the difference. Musicians had better short-term memory than nonmusicians for musical, verbal, and visuospatial stimuli with an effect size of, respectively, Hedges’ g = 1.08 (0.94-1.22) [large], 0.16 (0.02-0.30) [very small], and 0.28 (0.15-0.41) [small]. This work sets the basis for sound research practices in studies comparing musicians and nonmusicians, and contributes to the ongoing debate on the possible cognitive benefits of musical training.