In this study, 152 undisturbed soil samples from the 0–10 cm and 10–20 cm soil depths along two transects across the Taihang Mountain Region in northern China were investigated. The effect of soil properties (bulk density (BD), capillary porosity (CP), non-capillary porosity (NCP), soil organic carbon (SOC), and sand, silt, and clay contents) and environmental properties (land use type, topographical location, and elevation) on the K_s was determined. Moreover, the direct and indirect effects of these factors on the K_s were determined using the structural equation modeling (SEM). The results show that the K_s of the 0–10 cm soil depth (4.48 m/d) is significantly (p < 0.05) higher than that of the 10–20 cm soil depth (3.31 m/d), suggesting more rapid infiltration in the surface soil depth. The coefficients of variance (CVs) are 0.75 and 1.03 for the surface and sub-surface depths, respectively, indicating moderate and strong variations in the K_s in the two depths. The SEM revealed that most of the effects of the environmental properties on the K_s are indirect, accounting for 80.5% and 90.9% of the total effect in the surface and sub-surface soil depths, respectively. This demonstrates that the composite soil property was the direct, but the composite environmental property was the indirect factors influencing the K_s. These results deepen our existing understanding of the factors controlling the K_s in mountainous regions.