To understand the importance of the response of soil microbial communities to the stress of heavy metals around mining areas by assessing the feedback of soil ecosystems in different soil habitats, this article selected different land use types (Mining area, Dressing area, Heap mine area, Tailings area and Vegetable field) and surface soil samples of different depths (0–10 cm, 10–20 cm, 20–30 cm) as the variables related to the mining activities in the Shizishan mining area in Tongling, Anhui Province, China. Soil physicochemical properties and heavy metal concentrations of the different land use types and soil depths were compared. Illumina MiSeq. 2500 Sequencing Technology was used to analyze the abundance and structural diversity of the microbial community in soil samples. The relationship between mine soil pollution characteristics and microbial community were investigated. The results showed that soil physicochemical properties and heavy metals significantly affected the microbial community. The microbial community structure was significantly variable in vertical soil depth-layer habitats. The relative abundance (1%) of the soil microbial community at the phylum level was represented by a total of 14 phyla, where the two most dominant phyla were Proteobacteria (41.71%) and Firmicutes (20.44%). The two bacteria were positively related with Cu, Zn, Pb, and pH but negatively associated with soil organic matter (SOM), available potassium (AK), and moisture content (MC). Therefore, Proteobacteria and Firmicutes were highly resistant to heavy metals. These results increased our understanding of microbial variation and assembly pattern under different land use types in heavy metals contaminated mining soils.