The cultivation of leguminous green manure (LGM) crops in place of bare fallow in summer enhances soil fertility, which can positively impact soil life in the agroecosystem. An improvement in soil life, in turn, can promote soil fertility and agricultural sustainability. Based on an ongoing field trial in the Loess Plateau, China, we investigated the long-term effects of LGM crops and nitrogen (N) fertilization on soil mesofaunal community and analyzed the interrelationships among the various mesofaunal groups and soil fertility under dryland conditions. The results showed that the long-term growth of the LGM crops significantly increased the soil fertility and the abundance, richness, and diversity of the soil mesofaunal community compared to the fallow system. A total of 14 mesofaunal groups were identified only in the LGM system, mainly due to the high root biomass of the LGM crops. Among the three tested LGM crops, soybean and Huai bean performed better than mung bean in providing growth benefits and stable abiotic environments for the soil mesofaunal community. In addition, the long-term application of N fertilizer had an overall negative effect on the soil mesofaunal community under dryland conditions. Moreover, the abundance, richness, and diversity of the soil mesofaunal community were strongly correlated with soil fertility indicators. Hence, the analysis of the soil mesofaunal community can be used as a bioindicator of soil fertility in agroecosystems. The long-term growth of the LGM crops instead of bare fallow efficiently built a more sustainable and fertile soil in the agricultural ecosystem, thus providing a theoretical basis and guiding field management strategies for the development of sustainable agriculture in local and other similar dryland regions.