The increasing car usage has caused a series of social and environmental problems. Most previous studies examined the general correlations between the determinants and the usage pattern of private car by establishing global regression models. These models ignore the variation of travel behavior across temporal and spatial dimensions, which possibly leads to wrong bias and overlooks certain momentous details. Considering this research gap, this paper explores the influence of determinants on the time-of-day car usage pattern, using the multi-source data collected from the city of Kunming, China. It applies a geographically and temporally weighted regression (GTWR) model to investigate the heterogeneous relationship between on the hourly mode share of private car and the factors of individual attributes and built environment. The empirical results show that the GTWR model produces better goodness-of-fit compared with global regression model, as well as geographically weighted regression (GWR) model. It suggests that most explanatory variables are spatiotemporal non-stationary associated with hourly car share. A visualization method is applied to analyze the temporal variation of the coefficients of socio-demographic attributes and built environment variables, and to reveal the spatial distribution of the effects of determinants during peak periods. The empirical results show that both of selected the socio-demographic and built environment attributes have heterogeneous effect on the car share in time and space dimensions. For example, the proportion of college or college above presents stronger negative effect on the share of private car in the afternoon peak than in the morning peak, the negative impact of the densities of built environment (i.e., road, population, and bus stop) is more significant in underdeveloped area. Finally, insightful policy recommendations for different urban areas are presented to encourage the reduction of car usage and alleviate urban traffic problems.