Cities have been playing significant role in promoting energy efficiency and driving sustainable development. Energy consumption in different cities usually exhibit different characteristics. It is therefore important to evaluate and compare city-level energy efficiency performance by considering the impact of heterogeneity. This study employed stochastic frontier analysis to assess the spatial and temporal disparity of energy efficiency in Chinese cities in 2005–2015. The energy efficiency disparity across different cities was found to be significant, with energy efficient cities mainly located in more developed regions. We introduced a Mean-Variance Classification approach to address the effects of city heterogeneities and improve assessment reliability. This classification showed better discriminating power through narrowing the gap within groups and widening the gap across different groups. The empirical results reveal that mature cities with steady and high energy efficiency may influence and guide energy efficiency improvement of surrounding areas. To achieve energy efficiency improvement quickly, more attention should be paid to the promising cities with significant energy efficiency fluctuations.