Southern China and Southeast Asia witnessed some of their most significant economic and social changes relevant to human land use during the Late Holocene, including the intensification and spread of rice agriculture. Despite rice growth being associated with a number of earth systems impacts, how these changes transformed tropical vegetation in this region of immense endemic biodiversity remains poorly understood. Here, we compile a pollen dataset incorporating ∼150,000 identifications and 233 pollen taxa to examine past changes in floral biodiversity, together with a compilation of records of forest decline across the region using 14 pollen records spanning lowland to mountain sites. Our results demonstrate that the rise of intensive rice agriculture from approximately 2,000 y ago led not only to extensive deforestation but also to remarkable changes of vegetation composition and a reduction in arboreal diversity. Focusing specifically on the Tertiary relic tree species, the freshwater wetland conifer Glyptostrobus (Glyptostrobus pensilis), we demonstrate how key species that had survived changing environmental conditions across millions of years shrank in the face of paddy rice farming and human disturbance.