In China, the upstream Hangjing River (HJR) is an important surface water source for the South-to-North Water Transfer Project (SNWTP) because it has abundant water and better quality, which plays a crucial part in national drinking water safety and ecological sustainable development in China. This work was aimed at comprehensively studying the pollution status of heavy metals in sediments upstream of the HJR and apportioning sources. A total of 33 superficial sediment samples were collected from the main upstream of the HJR, which is located in Shaanxi and Hubei Provinces in China. The total concentrations of Zn, Cr, Ni, Cu, Cd, Pb, and As were analysed by using inductively coupled plasma mass spectrometry (ICP-MS). A standardized method was used to establish the environmental geochemical baseline concentrations (EGBCs) of the selected HMs as an alternative for local geochemical background values (LGBVs) to use for HM pollution assessment of sediment. The pollution status and potential ecological risk were assessed with the geochemical baseline factor pollution index (Ki), potential ecological risk index (RI), and sediment quality baseline evaluation (SQBEs). The source apportionment of heavy metals was performed by correlation analysis and principal component analysis (PCA). This study showed that the EGBCs of Zn, Cr, Ni, Cu, Cd, Pb, and As were 63.81 mg/kg, 23.47 mg/kg, 46.52 mg/kg, 35.26 mg/kg, 0.26 mg/kg, 6.97 mg/kg, and 6.76 mg/kg, respectively. The results indicated that Cd was the most common heavy metal pollutant. Mineral mining and smelting activities were the main sources of heavy metals in the sediments of the upstream HJR. Special attention should be paid to Cd, while Ni pollution cannot be ignored, especially in biotoxic effects. The high total Cd content affected individual and overall ecological risk indicators, indicating that Cd could represent a considerable ecological risk for upstream HJR.