Modeling takeover time based on non-driving-related task attributes in highly automated driving
Introduction
Automated assistive technologies have gradually been implemented in commercial cars over the past decade (Vollarath, Schleicher, and Gelau, 2011). Hence, fully automated driving systems will provide total control of driving tasks at all times in the near future. However, drivers are required to “be able to reengage control of the vehicle by switching from autonomous driving to manual driving” during partially automated driving, which is defined as Autonomous Driving Level 3 by the National Highway Traffic Safety Administration (NHTSA, 2013; Merat et al., 2014). Lu et al. (2016) defined the transition of control as the process by which an automated system switches from one driving state to another. Transitions of control occur from both vehicle to driver and driver to vehicle. The main factor in the transition from driver to vehicle is the driver's willingness to transfer control and use the automated system (Brandenburg and Skottke, 2014). To investigate important issues related to the transition from vehicle to driver, a more detailed analysis is required. Complex research topics such as lead time, time required to reengage the driving task, driving performance after switching to manual driving, request type, driver responsibility, and level of engagement in the driving task prior to transition of control are of particular interest (Gold et al., 2013, 2016; Jeon 2019; Larsson, 2017; Lee et al., 2020; Li et al., 2020; Lu et al., 2016; Sanghavi et al., 2020; Zeeb et al., 2016; Eriksson and Stanton, 2017; Yoon and Ji, 2019; Yoon et al., 2019; Yun and Yang 2020; Zeeb et al., 2015).
Transition of control from an automated driving system to the driver is defined as “takeover” (Gold et al., 2013; Zeeb et al., 2015; Melcher et al., 2015). Takeover can be classified into two categories based on whether the transfer of control is planned or unexpected. Planned transition of control refers to a predictable transition of control, which is easier to design compared to an unexpected transition (Merat et al., 2014; Naujoks et al., 2014). Unexpected transitions of control occur when the system experiences an error, emergency, or unpredictable system limitation. These unexpected and unpredictable requests necessitate time budget constraints for drivers to reengage control of the vehicle (Gold et al., 2013). Takeover is a relevant issue because drivers may engage in non-driving-related tasks (NDRTs) instead of maintaining vigilance and continuing to monitor the automated system (Banks and Stanton, 2017; Carsten et al., 2012). Both timing issues and exact and precise control of the vehicle following reengagement of control are important because a slight delay can lead to a critical situation (Gold et al., 2016; Jeon 2019; Lee et al., 2020; Lu et al., 2016; Sanghavi et al., 2020; Zeeb et al., 2016; Eriksson and Stanton, 2017; Yoon and Ji, 2019; Yoon et al., 2019). Additionally, cognitive aspects that influence manual driving following reengagement of control are considered to be relevant to this topic (Telpaz et al., 2015; Clark and Feng, 2015; Louw et al., 2017; Lu et al. 2017).
An increasing amount of research has focused on driver interactions with NDRTs in highly automated driving (HAD). It has been reported that NDRTs leads to the deterioration of driving performance (Merat et al., 2012) based on longer takeover time (TOT; Eriksson and Stanton, 2017). Gold, Happee, and Bengler (2017) stated that the complexity of an NDRT increases the time required to avoid a collision. However, when modeling TOT, these factors were not as significant as other variables, such as driver age, time budget, and traffic density. Mok et al. (2017) researched unstructured transitions of control with lead times of 2, 5, and 8 s when engaged in an NDRT consisting of playing a game on an iPad. The results indicated that 5–8 s is necessary for takeover and approximately 15% of participants attempted to pause the game before switching to the driving task, although they were presented with an emergency transition of control. Miller et al.’s (2015) research compared the effects of NDRTs (video watching and reading) using a mounted entertainment system on the vigilance of an automated system in HAD in terms of takeover performance. No significant difference was observed between the two NDRTs in terms of collision avoidance time, but a significant difference exists in the time required to switch to the driving task. Previous articles on this topic have discussed several aspects of NDRTs, such as cognitive load, physical engagement, and visual distraction. A comprehensive perspective that considers all of these aspects is necessary.
According to previous studies, the takeover process can be divided into different sub-processes based on the characteristics of each process during the transition of control. Zeeb, Buchner, and Schrauf (2015) proposed a model for the process of takeover of control, indicating that the cognitive processes of information and action selection are the key processes during takeover. During the cognitive process stage, drivers are required to perceive driving conditions and hazards, which affect their decisions regarding actions. In contrast, motor readiness represents reflexive reactions that are influenced by secondary tasks being performed during automated driving. Naujoks et al. (2017) presented a takeover process model for drivers when switching from NDRTs to a manual driving task. According to their model, drivers must transition through three reconfiguration states related to their sensory, motoric, and cognitive states. Therefore, the transition of control proceeds through four stages: 1) the maintenance of fitness to drive during HAD, 2) being alert to the importance of transition of control when a takeover request occurs, 3) disengagement from NDRTs to become ready to assume control of the vehicle, and 4) accommodation of resources for manual driving (Naujoks et al., 2017). By combining the various factors described above, we propose a conceptual model to explain the takeover process in HAD based on NDRT attributes.
The motor reaction stage refers to reflexive physical actions to prepare the body for driving. The mental reaction stage refers to the cognitive process in which drivers must process information gathered from their environments and make decisions to engage in the driving task. These processes of takeover of control are influenced by the driver state during automated driving. In other words, NDRTs performed by drivers during automated driving are expected to influence the transition of control to manual driving. Therefore, in this study, a quantitative approach was adopted to model the TOT of drivers and the effects of NDRT attributes on the reengagement of control processes (i.e., motor readiness, gaze time (GT), perceptual analysis, and response selection time). We categorized NDRT attributes into three groups: physical, gaze, and cognitive attributes. Physical attributes refer to the physical states of drivers. Based on previous research indicating that driver motor reactions for preparing to drive are reflexive reactions (Zeeb et al., 2015), we focused on analyzing the state of the body according to the hand in use, object of interaction, and active or passive interaction. The gaze attribute refers to where visual attention is directed when performing NDRTs. Finally, the cognitive resource attributes, which are considered before switching tasks, influence the time required to reassume control of the vehicle. Additionally, the level of engagement with NDRTs was considered as a cognitive attribute based on the argument presented by Horrey et al. (2017) that the degree of engagement in NDRTs influences driving performance.
We consider the effects of NDRT attributes on each stage of the takeover process. Based on previous research, we propose a model for the process of takeover in HAD and investigate which NDRT attributes influence each stage of takeover. We focus on the relationships between the NDRT attributes for each stage of the takeover process, which are classified as physical, visual, and cognitive attributes. Based on the effects of NDRT attributes on the motor and mental reactions of drivers, a prediction model for the time required for a driver to regain control of a vehicle is presented.
Section snippets
Modeling takeover time
In our conceptual framework for TOT, we categorized the transition of control based on reflexive reactions and mental reactions (Zeeb et al., 2015). Reflexive reactions are expected to be affected by the physical attributes of NDRTs. Wandtner, Schöming, and Schmidt (2018) observed that the modality of an NDRT is a relevant factor for TOT. The time required for participants to reach the steering wheel, indicating their physical readiness to drive, was longer when they were engaged in NDRTs with
Modeling
A multiple linear regression model was selected to predict the time required for drivers to reengage with the driving task after performing an NDRT in HAD. The regression model developed in this study predicts responses based on explanatory factors with several levels. Therefore, for analysis, we expressed these factors using dummy variables. Variance inflation factors (VIFs) were analyzed to avoid correlations among predictors (i.e., multicollinearity), where VIF values above 10 indicated that
Participants
In our model validation experiment, the same 30 participants from the previous experiment participated. This experiment was conducted in accordance with a protocol approved by the IRB with written informed consent. We conducted a driving simulator experiment in which participants were allowed to quit at any time when they experience motion sickness.
Experiment design and procedure
To validate the developed model, we conducted an experiment on the reengagement of control in HAD while considering 10 different NDRTs. A
Discussion and conclusions
This study aimed to develop a novel method for analyzing NDRT characteristics that influence driver states during automated driving and to evaluate the effects of the corresponding attributes on the process of taking over control of a vehicle based on takeover time. We presented a conceptual framework for takeover time based on previous studies, where we divided the types of reactions into reflexive and mental (Zeeb et al., 2015). We hypothesized that the physical aspects of NDRTs would
Declaration of competing interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This study is part of Ph.D. dissertation of the first author under the supervision of the corresponding author.
References (39)
- et al.
Conditionally and highly automated vehicle handover: a study exploring vocal communication between two drivers
Transport. Res. F Traffic Psychol. Behav.
(2019) - et al.
Development of NASA-TLX (task load index): results of empirical and theoretical research
(1988) - et al.
Distraction and task engagement: how interesting and boring information impact driving performance and subjective and physiological responses
Appl. Ergon.
(2017) - et al.
Effects of different non-driving-related-task display modes on drivers' eye-movement patterns during take-over in an automated vehicle
Transport. Res. F Traffic Psychol. Behav.
(2020) - et al.
Coming back into the loop: drivers' perceptual-motor performance in critical events after automated driving
Accid. Anal. Prev.
(2017) - et al.
Human factors of transitions in automated driving: a general framework and literature survey
Transport. Res. F Traffic Psychol. Behav.
(2016) - et al.
How much time do drivers need to obtain situation awareness? A laboratory-based study of automated driving
Appl. Ergon.
(2017) - et al.
Take over! A video-clip study measuring attention, situation awareness, and decision-making in the face of an impending hazard
Transport. Res. F Traffic Psychol. Behav.
(2020) - et al.
Take-over requests for automated driving
Procedia Manufacturing
(2015) - et al.
Transition to manual: driver behaviour when resuming control from a highly automated vehicle
Transport. Res. F Traffic Psychol. Behav.
(2014)
Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts
Transport. Res. F Traffic Psychol. Behav.
The effects of takeover request modalities on highly automated car control transitions
Accid. Anal. Prev.
What determines the take-over time? An integrated model approach of driver take-over after automated driving
Accid. Anal. Prev.
Is take-over time all that matters? The impact of visual-cognitive load on driver take-over quality after conditionally automated driving
Accid. Anal. Prev.
Automobile Automation – Distributed Cognition on the Road
Switching from Manual to Automated Driving and Reverse: Are Drivers Behaving More Risky after Highly Automated Driving?
Control task substitution in semiautomated driving: does it matter what aspects are automated?
Hum. Factors
Semi-autonomous vehicles: examining driver performance during the take-over
Takeover time in highly automated vehicles: noncritical transitions to and from manual control
Hum. Factors
Cited by (31)
Development and classification of autonomous vehicle's ambiguous driving scenario
2024, Accident Analysis and PreventionDriver behavior and mental workload for takeover safety in automated driving: ACT-R prediction modeling approach
2024, Traffic Injury PreventionTakeover quality prediction based on driver physiological state of different cognitive tasks in conditionally automated driving
2023, Advanced Engineering InformaticsDevil in the details: Systematic review of TOR signals in automated driving with a generic classification framework
2022, Transportation Research Part F: Traffic Psychology and BehaviourHow drowsiness and distraction can interfere with take-over performance: A systematic and meta-analysis review
2022, Accident Analysis and PreventionSafety at first sight? – Manual drivers’ experience and driving behavior at first contact with Level 3 vehicles in mixed traffic on the highway
2022, Transportation Research Part F: Traffic Psychology and BehaviourCitation Excerpt :The introduction of conditionally automated vehicles (CAVs; SAE Level 3, SAE International, 2021) on highways will result in mixed traffic where manual drivers in non-automated, assisted, or partially automated vehicles (up to SAE Level 2, SAE International, 2021) will have to share the road these new Level 3 vehicles. Over the last decade, human factors research focused primarily on the passenger perspective on the highway (Tabone et al., 2021), including the investigation of passenger comfort (e.g., Bellem et al., 2018; Cramer et al., 2020; Dillen et al., 2020), and engagement in non-driving related tasks during automated driving (e.g., Clark & Feng, 2017; Hecht et al., 2020; Naujoks et al., 2017; Shi & Frey, 2021; Wörle and Metz, 2020), as well as transition times and quality in take-over situations (e.g., Eriksson & Stanton, 2017; Merat et al., 2014; Naujoks et al., 2019; Petermeijer et al., 2017; Vogelpohl et al., 2018; Yoon et al., 2021). In contrast, human factors research has so far neglected the perspective of manual drivers in mixed traffic although Level 3 systems are soon to be introduced on highways in Germany (e.g., Audi, 2017; Daimler, 2019, 2021).