The quality of food crops and human health is threatened by heavy metals (HMs) accumulated in farmland soils for a long time. In this study, we selected 148 soil samples randomly from farmlands in a region featuring abandoned lead–zinc (Pb–Zn) mining activities with the aim to quantify the pollution risk and identify potential sources of heavy metals, based on a case in the southwestern of China. The median contents of metals, such as Pb, Zn, Cd, As, Cu and Cr, are above the background values for Chinese soils and prescribed pollution threshold guide values (GB15618-2018), except Hg and Ni. The farmland soils in sites surrounding areas with previous Pb–Zn mining and smelting activities were classified as seriously polluted. Pollution sources were evaluated using GIS-based geostatistical methods, multivariate statistical analyses and positive matrix factorization (PMF) modeling. Four sources were quantitatively apportioned, which were industrial sources such as mining and smelting (53.1%), agricultural practices (11.6%), natural source (21.6%) and other industrial sources such as electroplating (13.7%). The potential risks of contamination associated with the heavy metals were evaluated using several indices including the Nemerow, geoaccumulation (I_geo) and ecological risk (RI) indices. Based on the I_geo index, As and Pb were the most severe pollutants among all of those measured. With the combination of the potential ecological risk index (RI) and human health risk (HHR) assessment models, the ecological risk and HHR from different sources were analyzed quantitatively. Industrial activities such as mining and smelting were the greatest contributors to ecological risk, non-carcinogenic risk and carcinogenic risk, accounting for about 86.9%,73.9% and 81.9%, respectively. Additionally, the health risks of children were more serious relative to those of adults from the perspective of non-carcinogenic and carcinogenic risks.