This paper investigates two fundamental questions related to the social impact of high-speed rail (HSR): (1) does the development of HSR infrastructure lead to the exclusion of certain social groups? and (2) what factors exclude certain users from using HSR? Based on a comprehensive survey of 4,924 passengers who chose conventional rail services in China, both binary and multinomial logit analyses were conducted to examine to what extent the mode choice of passengers who departed from Chongqing was affected by various factors, including socioeconomic and train service conditions. The analyses reveal that factors such as fare, travel habits, trip distance, and onboard amenities play significant roles in whether passengers choose conventional rail over HSR. Conversely, the study shows that lower incomes and education levels are associated with higher odds of choosing conventional trains over HSR by 24.1% and 25.3%, respectively. Furthermore, the study shows that the preference of conventional rail to HSR varies among different regions in China. In particular, people were found to prefer less to choose conventional trains in the west and central regions than the east region when the HSR service becomes available. Such a pattern suggests that HSR as a new mode of intercity travel, has become a preferred mode for many people, even for those in the less developed regions in China. However, some passengers from less developed western provinces were also found to have a stronger sense of being excluded from using HSR, given the concerns of high fares and personal perception of cities that the system serves. Overall, this study provides important implications for planning and policy-making to achieve sustainable operations and the development of the passenger rail system.