Phosphorus (P) is considered an indispensable and finite resource that limits crop yields, and soil P is distributed heterogeneously in space. However, a systematic analysis of previous studies indicates that there are large gaps in our knowledge regarding the spatial patterns of soil P due to the lack of sufficient field observations, especially at larger scales. Thus, we explored the spatial patterns of soil total P (TP) and Olsen-available P (AP) in the Hexi Corridor, which is an important commodity grain base for China where large quantities of chemical P fertilizers were applied. We analyzed the spatial variability of soil P using semivariogram and Anselin Local Moran’s I analysis at the regional and local scales. We explored the driving factors that influence the variances in soil P using principal component regression and Johnson’s relative weights. Overall, soil TP was at a “low” level (0.55 g/kg in 0–20 cm and 0.49 g/kg in 20–40 cm) and AP was at the “lowest” level (2.54 mg/kg in 0–20 cm and 1.73 mg/kg in 20–40 cm). The soil TP and AP levels tended to be similar within land use types and they varied according to the following decreasing order: farmland > grassland > forestland > bare land. The soil TP and AP both exhibited strong spatial dependence (nugget ratio = 4.4–8.0%) within large ranges (132.0–222.0 km). At the local scale, Anselin Local Moran’s I analysis indicated the existence of some clusters and outlier areas, and land use determined the local disparities in the soil P contents. Ordinary kriging (OK) interpolation was used to obtain unbiased estimates of the soil TP content (mean absolute error (MAE) < 0.002 g/kg) and AP content (MAE = 0.027 mg/kg), but the predictions of the soil P at local positions were not satisfactory, where the normalized root mean square error varied from 78.8 to 84.8%. Land use and normalized difference vegetation index (NDVI) were major factors that contributed to the spatial variability in the soil P. These results indicate severe P limitation in the northwestern China and land use dominates the spatial variability of soil P at both local and regional scales. Our finding provides information that is relevant to assess fertilizer application, soil P predication and vegetation restoration.