Desertification degrades soil health and severely reduces crop productivity. Conventional tillage practices can amplify these problems in arid and semi-arid regions. For semi-arid regions in Inner Mongolia, China, the effects of introducing different tillage practices such as subsoiling (SS), straw mulching (SM), and no-tillage (NT) on otherwise long-term conventional tilled maize monoculture systems, were examined in the context of sandy soil bacterial and archaeal community diversity. Results showed that after three to four years of introduction, subsoiling and conversion to conservation tillage practices had no immediate effect on the alpha-diversity of the soil microbial communities. The beta-diversity of the soil microbial communities was less affected by the introduced tillage practices than by the growing season conditions, soil moisture, total nitrogen, soil macro-aggregate, and organic matter. Importantly, the introduced tillage practices had a notable effect on soil microbial communities associated with nitrogen (N) cycling processes, especially N fixation, nitrate reduction, nitrification, and denitrification. In particular, several years of tillage change from long-term conventional tillage enhanced the abundance of KEGG orthologs (KOs) associated with N fixation function involving species in Rhodoplanes, Nitrospira, Skermanella, and Rhizobium according to PICURSt prediction. Rhodoplanes spp. are involved in nitrate reduction and denitrification processes, and Nitrospira spp. associated with nitrite oxidization. We conclude that, for maize monoculture systems in semi-arid sandy soils, the soil bacterial and archaeal communities associated with many beneficial N cycling processes can be significantly impacted by only three to four years of introduced conservation and subsoiling tillage practices.