Modeling travel behavior inter-relationship is important to traffic planning and management as well as spread modeling of infectious disease and information. Our ability to understand inter-traveler behavior has remained unsatisfactorily limited due to the lack of empirical travel-together data. Taking advantage of access to a large-scale smart-card dataset from the Beijing metro network, we investigate a KS test-based method to identify group travel and study the spatial–temporal distribution of group travel as well as their interaction with socio-economic attributes. Origin and destination stations, departure time, and travel dates are separately classified into several types according to the passenger flow adopting a tensor factorization technique. The results suggest that temporally passengers tend to travel in groups in the afternoon and on the weekend; spatially areas with moderate house prices are the least source for group demands, while areas with a relatively low house price are the most favorite place for group trips. This novel discovery can help to understand the impact of socio-economic attributes on intra-urban group/passenger flow movement.