Plant functional traits shape many ecosystem processes, including carbon sequestration in vegetation and soil. The shifts of trait-based indices can reflect changes in plant resource-use strategies during successional processes. The aim of this study was to determine how plant functional diversity affects carbon sequestration during secondary succession in semi-arid grassland, and how the limiting environmental factor (that is, soil water) affects these processes. We investigated 16 plots at three sites with different stand ages and climate conditions in the semi-arid area of the Loess Plateau, China. We measured the key plant functional traits: plant height; leaf dry matter content (LDMC); specific leaf area (SLA); leaf carbon (LC), nitrogen (LN), and phosphorus contents (LP); and ecosystem carbon components: aboveground biomass (AGB); root biomass (RB); and soil organic carbon stock (SOC). The community weighted mean (CWM) of LDMC increased and SLA and LN decreased with stand age at a wetter site, indicating that plants’ resource-use strategy might shift from acquisition to conservation during succession. The exception of the shifting of CWM_SLA deviated from the general trend during succession can be attributed to the appearance of slow-growing species in the early successional stage at a drier site. The structural equation models (SEM) showed that stand age, soil water content (SWC) and functional diversity indices accounted for 55.7%, 53.2%, and 56.5% of variations in AGB, RB, and SOC during succession, respectively. The interplays of stand age and SWC were regulated by functional diversity indices, especially the CWMs of plant height and SLA, functional variance (FDvar) of plant height and functional evenness (FEve). Carbon stocks accumulated faster at wetter sites. Our results indicate that the mass ratio and niche complementary hypotheses are not contradictory in explaining relationships between community functional diversity and ecosystem carbon stocks during the restoration of semi-arid grasslands, and that SWC affects the progress of succession and carbon sequestration.