Heavy metal pollution of soils has been worsening increasingly in China, which brings significant health risk to human, it is critical to investigate the sources of heavy metals in agricultural soils and explore the effects of heavy metal accumulation in crops. In this paper, the sources of cadmium (Cd) and their effects on Cd accumulation in soil and rice grown on urban farmland in Changzhutan were investigated. Among the main Cd sources (irrigation water, commercial fertilizer, and atmospheric deposition), the input flux of atmospheric deposition accounted for 76.36%–98.25% of total input flux, significantly higher than the input fluxes of irrigation water and commercial fertilizer. Manure fertilizer was also an important source of Cd in livestock breeding areas. The accumulation behaviors of Cd in soils and plants presented significant spatial variation among the study areas. Higher Cd input flux from atmospheric deposition resulted in higher Cd bioavailability in soil and more Cd accumulation in rice, and the newly deposited Cd contributed 7.35–41.23% in rice tissues. The use of manure fertilizer increased the soil pH and amount of available Cd in soil, as well as the accumulation of Cd in rice roots. Based on sequential extraction, acid-extractable Cd accounted for approximately 52.54%–61.88% of total Cd in atmospherically deposited particles in the study area, resulting in a high proportion of acid-extractable Cd in soil. This study provides useful reference data on the sources of Cd and its bioavailability in soil and rice.