Navigating with Augmented Reality – How does it affect drivers’ mental load?
Introduction
Navigating is an elementary driving task. It is a spatial task as the driver needs to orient himself in the environment. Drivers have to plan their routes and estimate costs and risks. For navigating in a foreign environment, or in ambiguous navigation scenarios, knowledge based behaviour (Rasmussen, 1983) is necessary. This requires the driver's resources and therefore can be cognitively demanding. However, it is possible to support the driver through improved displays. Because information and digitization in general, available traffic-related information in the vehicle is increasing, but information should be easy, readily accessible, and presented comprehensively without distracting the driver. For example, heads-up displays (HUD) enable drivers to receive driving related information within their field-of-view. Hence, information is readily available enabling concurrent scanning of the information provided and the environment (Prinzel and Risser, 2004) and minimizing drivers' gazes away from the street. However, the information presented in a HUD does not necessarily relate to the environment. Thus, drivers have to map the 2D information onto the real driving situation. This demands drivers' attention and may lead to navigation and driving errors.
Advanced HUDs are able to present Augmented Reality (AR) information, which is directly related to the driving situation. For example, while following the navigation arrow presented through an AR display, the driver gets the impression that the path-to-follow is virtually marked on the street. This might result in the driver easily and quickly understanding AR information. As the augmented information is correctly superimposed on the relevant objects in the environment, virtual and real objects are closer. Thus, the driver no longer has to switch between the relevant information and the real traffic situation (Kim and Dey, 2009). Moreover, with AR the driver does not have to map the virtual information onto the environment (Pfannmüller, 2017). Based on these advantages, AR is often applied in navigation. AR navigation information is intuitive to understand and facilitates the navigation task resulting in reduced navigation errors (Bauerfeind et al., 2019; Israel, 2012; Kim and Dey, 2009). In the Kim and Dey (2009) study, participants showed enhanced driving performance (e.g., completion time, number of missed turns) while driving with AR compared with digital maps shown on a center information display. This is in line with Medenica et al. (2011), who also found that participants showed an improved driving performance (i.e., lane position, steering wheel angle) while driving with an AR display for a navigation task rather than with a standard map-based navigation device or an egocentric street view navigation device. The egocentric street view navigation device was an LCD display presenting the driving situation from the driver's perspective including AR navigation information.
In the literature, several authors claim that AR navigation information requires less mental effort in understanding and interpreting information compared with navigation information in the HUD (Bengler et al., 2015; Israel, 2012; Kim and Dey, 2009; Pfannmüller, 2017; Pfannmüller et al., 2015). In the Medenica et al. (2011) study, participants felt less subjective workload while driving with navigation information presented in an AR display than any other navigation device, like map-based navigation. Most findings in the field of mental load levels with AR are based on subjective ratings. However, subjective data (questionnaire data) might be susceptible to distortions caused by social desirability. Furthermore, little is known about the effects on objective performance measures resulting from lower mental load level with AR. If the navigation task with AR (primary task) is less demanding, then the driver is more capable of performing better in other tasks (e.g., in a non-driving-related task [NDRT]) (Jahn et al., 2005). It must be closely observed whether AR information leads to reduced mental load resulting in enhanced performance.
The research presented here is aimed at examining whether AR information leads to objective performance improvements in terms of a lower mental load level compared with HUDs. AR information is likely to demand the driver less regarding visual, spatial resources compared with information in a HUD. Previous results have shown that AR information leads to earlier decisions in navigation tasks (cf. Bauerfeind et al., 2019). Research is needed into whether this results from a reduced mental load level with AR information. Moreover, whether a lower mental load level due to AR information eventuates in altered driving behaviour in comparison to a HUD must be analysed. This research focus is on drivers’ mental load level and driving behaviour while navigating with AR information compared with a HUD. It is expected that AR information demands less mental load than HUDs, especially in ambiguous navigation situations (e.g., turning scenarios with many possible turns, which are very close to each other). In total, 59 participants had to find the target intersection in ambiguous navigation situations in an urban area. Additionally, drivers had to accomplish a non-driving-related task. This NDRT was used to measure the mental load while the driver was identifying the target intersection supported by visual navigation information presented in the AR display or in the HUD. In the study, the performance in the NDRT, driving data, and subjective data while approaching the target intersection were analysed.
Section snippets
Display types for the navigation task
Fig. 1 presents the two display types (HUD and AR display) used for the navigation task. In the study, the HUD had a perceived projection distance of approximately 1–2 m and was presented in 2D. The driver received the navigation information 300 m before the target intersection in form of a straight blue arrow (Fig. 1a). The turning direction was presented to the driver 80 m before the target intersection (Fig. 1b). The bar graph next to the blue turning arrow in the HUD indicated the remaining
Data preparation
In total, 59 participants were analysed. The NDRT was used to measure the drivers’ mental load. Because of technical issues, 697 trials could be observed in this analysis. To prepare the NDRT data, speech recognising software was written in R to extract the response time. For the analysis, the response time is the time stamp when participants started answering. A hierarchical linear model with maximal random effects structure and models with reduced random effects structures (Barr et al., 2013)
Discussion
The aim of the driving simulator study was to examine differences in drivers' mental load in a navigating task using an AR display or a HUD. The study presented here confirms that AR information is more easily and quickly understood by the driver than a HUD. Results showed that AR information led to more confident driving behaviour than with the HUD. Furthermore, data revealed that AR information requires less of the drivers’ mental load in understanding and interpreting compared with HUDs.
Conclusion
This research was focused on the effects of AR information on drivers’ mental load in a navigation task. Overall, this study confirms that AR information is more easily and quickly understood by the driver than a HUD. Moreover, data showed that AR information led to more confident driving behaviour than with the HUD. Furthermore, fewer resources appear necessary while navigating with AR information. The reduced mental load level might result in safer driving with less inattention because the
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
We express our sincere thanks to Michael Wittkämper for implementing the AR display in the simulated world, to Lea Dingler for her support to collect the data and to Anne-Marie Wesseling for her support in data preparation. Gratefully acknowledged is Professor Stephen Arnott, who provided feedback on the manuscript.
References (18)
- et al.
Random effects structure for confirmatory hypothesis testing: keep it maximal
J. Mem. Lang.
(2013) - et al.
When does the driver benefit from AR information in a navigation task compared to a Head-Up Display? Results of a driving simulator study
- et al.
To see or not to see - innovative display technologies as enablers for ergonomic cockpit concepts. Ergonomic requirements, future mobility, future functionality
- (1922)
The speed-accuracy tradeoff: history, physiology, methodology, and behavior
Front. Neurosci.
(2014)Theorie und Praxis des Verfahrens der Kategorieunterteilung (KU)
Virtual Test Drive, vollständige Prozesskette für die Fahrsimulation
Potenziale eines kontaktanalogen Head-up Displays für den Serieneinsatz (Doctoral dissertation)
(2012)- et al.
Peripheral detection as a workload measure in driving: effects of traffic complexity and route guidance system use in a driving study
Transport. Res. F Traffic Psychol. Behav.
(2005)
Cited by (17)
Improving pedestrians’ navigation safety at night by enhancing legibility of foreground and background information on the display
2023, International Journal of Industrial ErgonomicsMeasuring workload effects of augmented reality head-up displays using detection response task
2023, Transportation Research Part F: Traffic Psychology and BehaviourCitation Excerpt :Many studies evaluated the effects of automotive HUDs compared to HDD, mostly resulting in beneficial effects for HUDs (Horrey et al., 2003; Kiefer, 1991, 1998; Kim et al., 2013; Lorenz et al., 2014; Sprenger, 1993). Although several studies also resulted in positive effects of AR-HUDs (Bark et al., 2014; Bauerfeind et al., 2019; Bauerfeind et al., 2021; Frémont et al., 2020; Kim et al., 2016; Medenica et al., 2011; Rusch et al., 2013; Schall et al., 2013; Wang et al., 2020), it remains unclear how AR-elements have to be designed to prevent potential negative side-effects like distraction and additional workload. This paper evaluated a methodology based on the DRT (ISO, 2016) for measuring the distraction and workload effect of three different display technologies: HDD, HUD, and AR-HUD.
Determining the impact of augmented reality graphic spatial location and motion on driver behaviors
2021, Applied ErgonomicsCitation Excerpt :Our results build upon previous work which has shown potential value of world-registered graphics by highlighting the importance of careful interface design: augmented reality alone does not inherently create a better experience for drivers. In fact, our results show that even for spatial location tasks, which are widely heralded as an ideal use case for world-registered AR graphics due to the need to integrate digital navigation instructions into the real world (e.g., Bauerfeind et al., 2021), integrating graphics alone does not improve performance. Graphics integrated into the world (rather than simply localized), must be further researched to fully understand the impact on end-users.
Effects of Focal Plane Distance on Perceptual Distance Matching with an Automotive AR-HUD
2023, ACM International Conference Proceeding SeriesEnhancing Perception of Risk Objects for Car Drivers Through Augmented Reality Glasses
2023, ACM International Conference Proceeding SeriesAugmented reality HUD vs. conventional HUD to perform a navigation task in a complex driving situation
2023, Cognition, Technology and Work