We collected 682 topsoil samples (0–20cm) from agricultural lands of Luhe County in East China, and analyzed the spatial distribution patterns and potential sources of four major heavy metals. High Pb and Cr were mainly in the southeast adjacent to the Yangtze River, and Cd were characterized by an increasing trend from northwest to southeast, while high Hg mainly occurred in the areas near downtown. Spatially-continuous sources dominated the soil heavy metal concentrations. Contributions of spatially-continuous natural source (soil parent material) to Cr and Cd were 97.0% and 77.7%, respectively, whereas contributions of spatially-continuous anthropogenic source such as diffuse pollution to Pb and Hg were 75.7% and 86.7%, respectively. The distance to factories was the most influential anthropogenic factor for localized anomaly patterns of Pb, Cd, and Cr, while the intensive agricultural land uses associated with the rapid urban expansion were particularly relevant to the anomaly patterns of Hg.