The heavy metal pollution level in soils is heavily affected by the soil particle size distribution. However, the heavy metal loss during particle size extraction and the effect of calcite on the heavy metals removal in terms of the particle size are unclear. In this study, the distribution of heavy metals (Cu, Cd, Cr, Co, Ni, Zn, Pb, U, and V) was determined in five particle fractions (> 2, 2–0.25, 0.25–0.02, 0.02–0.002, and < 0.002 mm) of two soil and one sediment samples collected from the floodplain of Dongchuan, Yunnan Province, Southwest China. The sampled floodplain soils were mainly composed of gravel and sand fractions (> 97%). The concentrations of all nine heavy metals in the sampled soils and sediment increase significantly with decreasing particle sizes. The maximal loss rate of Cd and Cu reaches 54% and 8.6%, respectively, which should be considered in the process of particle size fraction extraction in soils. The removal amount and removal rate of heavy metals in solution by pure calcite ranks in the order of Pb²⁺ > Cu²⁺ > Cr⁶⁺, while the removal rate of Pb (93.13%) is much higher than that of Cu (24.56%) and Cr (10.71%), which increase with the calcite particle size decreasing. The stabilization of carbonate minerals in soils is crucial for heavy metal pollution control in floodplain soils with high carbonate concentrations in Dongchuan, China.