Several studies support a plausible association of human exposure to polychlorinated biphenyls (PCBs) with type 2 diabetes mellitus. However, little is known about the potential link between circulating PCBs levels in early pregnancy and development of gestational diabetes mellitus (GDM). Here, we conducted a prospective study with a nested case-control design to determine the association of PCB exposure in early pregnancy with GDM risk. Concentrations of selected so-called ‘indicator PCBs’ were determined in sera of women in the first trimester of pregnancy via gas chromatography - high resolution mass spectrometry. Pregnant women were screened for GDM at 24–28 weeks of gestation using the oral glucose test (OGTT). GDM was defined based on the diagnostic criteria set by China Ministry of Health. Conditional logistic and linear regression models were employed to evaluate the association of PCB exposure with GDM risk and OGTT values, respectively. The odds ratios (OR) of PCB-28, PCB-52, and PCB-101 for GDM were 1.86 (95% CI: 1.05–3.27), 1.90 (95% CI: 1.28–2.82) and 1.85 (95% CI: 1.22–2.82), respectively. No statistical association was evident for other PCBs. However, after adjusting for confounders including some PCB congeners, only PCB-52 remained significantly associated with GDM with OR of 1.97 (95% CI: 1.27–3.07). Moreover, PCB-52 was positively associated with all blood glucose values of OGTT (p < 0.05). Our findings collectively suggest that serum levels of specific non-dioxin-like PCBs in early pregnancy disturb the glucose metabolism and increase the risk of GDM.