Heavy metals in lake sediment have become a great concern because their remobilization has frequently occurred under hydrodynamic disturbance in shallow lakes. In this study, heavy metals (Cr, Cu, Cd, Pb, and Zn) concentrations in the surface and core sediments of the largest freshwater lake in China, Poyang Lake, were investigated. Geostatistical prediction maps of heavy metals distribution in the surface sediment were completed as well as further data mining. Based on the prediction maps, the ranges of Cr, Cu, Cd, Pb, and Zn concentrations in the surface sediments of the entire lake were 96.2–175.2, 38.3–127.6, 0.2–2.3, 22.5–77.4, and 72.3–254.4 mg/kg, respectively. A self-organizing map (SOM) was applied to find the inner element relation of heavy metals in the sediment cores. K-means clustering of the self-organizing map was also completed to define the Euclidian distance of heavy metals in the sediment cores. The geoaccumulation index (I_geo) for Poyang Lake indicated a varying degree of heavy metal contamination in the surface sediment, especially for Cu. The heavy metal contamination in the sediment profiles had similar pollution levels as those of surface sediment, except for Cd. Correlation matrix mapping and principal component analysis (PCA) were used to support the idea that Cr, Pb, and Zn may be mainly derived from both lithogenic and human activities, such as atmospheric and river inflow transportation, whereas Cu and Cd may be mainly contributed from anthropogenic sources, such as mining activities and fertilizer application.