Phelan-McDermid syndrome (PMS) is characterized by a variety of clinical symptoms with heterogeneous degrees of severity, including intellectual disability, speech impairment, and autism spectrum disorders (ASD). It results from a deletion of the 22q13 locus that in most cases includes the SHANK3 gene. SHANK3 is considered a major gene for PMS, but the factors modulating the severity of the syndrome remain largely unknown. In this study, we investigated 85 PMS patients with different 22q13 rearrangements (78 deletions, 7 duplications). We first explored their clinical features and provide evidence for frequent corpus callosum abnormalities. We then mapped candidate genomic regions at the 22q13 locus associated with risk of clinical features, and suggest a second locus associated with absence of speech. Finally, in some cases, we identified additional rearrangements at loci associated with ASD, potentially modulating the severity of the syndrome. We also report the first SHANK3 deletion transmitted to five affected daughters by a mother without intellectual disability nor ASD, suggesting that some individuals could compensate for such mutations. In summary, we shed light on the genotype-phenotype relationship of PMS, a step towards the identification of compensatory mechanisms for a better prognosis and possibly treatments of patients with neurodevelopmental disorders.