An efficient transportation system is conducive to maintaining traffic flow and safety. Passenger flow forecasting (PFF), an area of traffic forecasting, is a key part of the efficient transportation system. In recent years, deep-learning-based models have led to extensive research on the different conditions in this field. Hence, model determination is the most suitable for a specific application would be a key advantage. To address this issue, a comparative analysis of nine typical deep network approaches, including recurrent neural network (long short-term memory (LSTM) and gated recurrent unit (GRU)), bidirectional-based RNN (BiLSTM and BiGRU), convolutional neural network (CNN) (CNN1D and CNN2D), convolutional LSTM, and hybrid network (CNN1D-LSTM and CNN1D-GRU), for hourly PFF has been conducted. This comparison utilized two datasets with hourly records of bus lines from Guangzhou, China. The comparison results show that the bidirectional-based models were slightly better than other candidates in terms of the values of the root-mean-square error, determination coefficient, and Theil coefficient, and had a lower individual error distribution than the others, both numerically and proportionally. Furthermore, the bidirectional-based models were different from the other models in terms of the Friedman test (a special case is that the BiLSTM and LSTM had no significant difference for Line 10 application). Besides, the results from model structure aspect indicated that the bidirectional-based models achieved better performance with stable and reliable model structure (measurement index: posterior error distribution) and less computational complexity (measurement index: number of floating-point operations and parameters). It is concluded that the bidirectional-based deep learning models are the preferential choice for hourly PFF.