We ask what effects a high case fatality rate pandemic could have on city growth. The Black Death killed 40% of Europe’s population between 1347 and 1352. Using a novel dataset on Plague mortality at the city level, we explore the long-run impacts it had on city growth. On average, cities recovered their pre-Plague populations within two centuries. However, aggregate convergence masked heterogeneity in urban recovery. We show that both of these facts are consistent with populations returning to high-mortality locations endowed with more rural and urban fixed factors of production. Land suitability and natural and historical trade networks played a vital role in urban recovery. Our study thus highlights the role played by pandemics and physical and economic geography in determining the relative size of cities in poorer countries.