Increasing evidence has suggested that physical activity may modulate gut microbiome composition. We investigated associations of long-term regular exercise with gut microbiota among middle-aged and older urban Chinese individuals. Gut microbiota was assessed using 16S ribosomal ribonucleic acid gene sequencing of stool samples from 2,151 participants from the Shanghai Women’s Health Study and Shanghai Men’s Health Study. Participants were free of cancer, diabetes, and cardiovascular diseases at the time of stool sample collection. Physical activity was assessed in repeat surveys between 1996 and 2015 using validated questionnaires. Regular exercise was defined as any type of leisure-time physical activity with a standard metabolic equivalent score >3.0. Stool samples were collected using the 95% ethanol method between 2015 and 2018 with an average of 3.0 years (SD = 0.9) after the latest exposure assessment. General linear regression and permutational multivariate analysis of variance were carried out to evaluate associations of microbial α- and β-diversity with regular exercise participation. Logistic regression and linear regression models were used to evaluate the prevalence and relative abundance of individual taxa in association with regular exercise. Regular exercise was significantly associated with β-diversity (Bray–Curtis and Jaccard dissimilarities, both false discovery rates = 0.03%, 0.12% and 0.09% variance explained, respectively) but not with α-diversity. Relative abundance of genus Ruminococcus was significantly lower among regular exercisers compared with nonexercisers (median relative abundance: 0.64% vs. 0.81%, false discovery rate <0.10). Further studies are needed to validate the findings from this study and evaluate health benefits of regular exercise on gut microbiota.