The smart city, as a new urbanization mode, provides a practicable initiative for achieving the win-win situation of economic development and environmental protection. This paper examines the influence of smart city construction on urban green total factor productivity (GTFP), explores the influence mechanism and further investigates the heterogeneous influence among the factors of urban scale, human capital and financial development, by conducting a quasi-natural experiment based on the first batch of pilot smart city in China and using panel data of 174 prefecture level cities over the period of 2005−2016. Our findings show that: (1) Smart city construction pilot significantly improves GTFP and green technological progress (GTP) in China, but its influence on GTE is not significant, which means smart city construction promotes GTFP mainly through promoting GTP. This conclusion is still valid after a series of robustness tests. (2) The estimation result of mediating effect models is that smart city construction promoted GTFP and GTP mainly through boosting urban technological innovation. The influence path of “smart city construction→technological innovation→GTFP” is verified. (3) Regarding city heterogeneity, cities with larger scale, richer human capital and higher level financial development have stronger positive effects on GTFP and GTP. These conclusions are of vital significance to improve the urban ecological environment and promote the high-quality development of urban economy.