Mining activities are well-known sources of potentially toxic elements (PTEs) pollution, which often jeopardize the biosphere, pedosphere, and hydrosphere. However, the soil and groundwater pollution caused by active private mining activities has long been neglected. This study investigated the occurrence of PTEs and cyanide (CN) in agricultural soils, mine tailings, and groundwater nearby the cyanide baths from a private gold mine in Hainan Province, southern China. Results indicated that concentrations of Pb, As, Cd, Hg, and CN in different soil depths and mine tailings were up to ten thousand mg/kg, and relatively higher content of As and Pb was detected in groundwater. The chemical forms of Cd, Pb, As, and Hg varied greatly in different soil depths; over 80% of Cd distributed in the water-soluble fraction, suggesting its higher mobility in soils, while approximately 60–90% of Pb, As, and Hg distributed in other chemical fractions, indicating relatively lower mobility in soils. The pollution indices also revealed the serious pollution and deterioration of site quality in this area. Human risk assessments also reflected a high non-carcinogenic/carcinogenic health risk in this area. The framework of integrated management strategies for private metal mines was proposed to mitigate PTEs pollution and reduce health risks.