Tea (Camellia sinensis L.) plantation appears to be a sustainable alternative to other crop types because it has the potential to improve nutrient cycling and the productivity in Southwest China. However, the changes of soil organic matter (SOM) quality during the long-term tea planting process have not yet been well documented, and it is difficult to reveal tea plantation effects on the chemical stability of SOM. In this work, the changes of SOM functional groups during tea planting were studied at five sites: four tea plantations with different durations of use (18a, 25a, 33a, 55a), and an adjacent abandoned land (0a), by analyzing the change of soil organic carbon (SOC) and the infrared spectroscopy and element composition characteristics of humic acids (HA), fulvic acids (FA) and humin (HM). Compared with abandoned land (0a), the content of SOC, HA, FA and HM in tea plantation soil increased significantly, and the content of SOC was higher in 25-year-old tea plantation, decreased significantly 33-year-old tea plantation. After tea plantation, the H/C and O/C of HA and HM decreased, and the aliphatic chain hydrocarbons in HA and HM molecules decreased relatively, and the aromatic structure increased relatively. The H/C and O/C in the soil HA, FA and HM were lower in the 25-year-old tea plantation. Tea plantation was beneficial to increase the content of SOC and humic substances, enhance the degree of soil humification and improve the quality of soil humus. The quality of soil humus was better and the effect of carbon sink was stronger in 25-year-old tea plantation. It is necessary to improve fertilization management to increase the exchange sites of SOM, so as to improve nutrient cycling, maintain the structure and fertility of tea plantation soil, and realize the sustainable utilization of regional tea plantation soil.