Task performance in a head-mounted display: The impacts of varying latency☆
Introduction
Head-Mounted Displays (HMDs) are worn on the head and close to the eyes, providing users with an image resulting in a virtual visual surround. Current technology tracks head motion to update displays to the users’ point of view in the virtual surround, as they make head movements [1]. Although commonly used for entertainment, they are increasingly being used by workers performing remote operations, maintenance, engineering, and simulations [2], [3]. HMD users routinely encounter anomalies such as delays in feedback, or occlusion of peripheral vision, which interfere with their visual perception.
System latency is the time it takes for a real-world event to be sensed, processed, and displayed to the user. Latency is commonly in the range of tens to hundreds of milliseconds (ms) and causes control problems for users. In virtual reality systems, latency has been shown to confound tasks where timing is critical to successful task completion, such as pointing and object motion tasks [4], catching tasks [5], and ball bouncing tasks [6]. In robotics, latency has an impact on teleoperation and remote manipulation because of visual display limitations and delayed feedback [7], [8]. Delayed feedback can slow response time of the virtual display user, thereby decreasing the user’s ability to perform tracking and pursuit tasks [9], [10]. Feedback delays create a conflict because users cannot reliably use feedback from their actions to correct their behavior. While the perceptual sensitivity to latency has been investigated (e.g., [11]), there is little research evidence to support a critical threshold at which latency no longer produces negative side effects in terms of simulator sickness and decrements in performance.
System latency in HMDs is normally reported as a constant value. However, research has shown that system latency varies, fluctuating in both rate and magnitude [12]. In inertial based tracking systems, head-tracking sensor errors lead to varying latency in visual displays [12], that is, as a head-mounted display (HMD) wearer moves their head, the visual images being displayed are delayed in varying amounts, depending on the moment-to-moment accuracy of the head tracking. It has been shown that varying latency is related to simulator sickness [13], [14]. However, there is no research, to our knowledge, investigating the impact of variability in latency on performance (calibration) in an HMD.
The purpose of this study was to determine how varying latency in a head-mounted display affects human performance. The hypotheses were divided into two areas: accuracy and time-to-hit of targets. We expected performance to be better in the constant latency condition because individuals are known to be capable of calibrating to constant perceptual perturbations. However, when perturbations vary, individuals cannot calibrate and therefore performance will be more affected.
Section snippets
Participants
Thirty participants were recruited from the student, staff and faculty population of Clemson University via flyers and the Department of Psychology human subject pool website. Where relevant, some participants were given course credit. All participants were paid $15 for their first session and $35 for their second session. To be eligible, participants could not have any history of brain, heart, stomach, eye (other than contacts), or inner ear problems, or be pregnant. This research complied
Results
Of the 30 (15 male) participants in the study, one female participant did not return for the second experimental session and therefore her data were omitted from analyses. The demographics of the remaining 29 participants are shown in Table 3.1. There were no differences noted between the male and female groups for age, race or handedness.
Discussion
The purpose of this study was to determine how varying latency in a head-mounted display affects human performance. More specifically, the study examined the effects of system latency on accuracy and time needed to score a hit. Overall, the findings indicate that human performance is lower in the presence of varying latency.
A main effect of condition on accuracy was observed. Participants were less accurate in the varying latency condition than in the constant latency condition. This supports
Funding
This work was supported by the Clemson University Human Factors Institute for subject payments and supplies.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Michael L. Wilson is a research psychologist at the U.S. Army Aeromedical Research Laboratory. He received his PhD in human factors psychology from Clemson University in 2016.
References (28)
The visual effects of head-mounted display (HMD) are not distinguishable from those of desk-top computer display
Vision Res.
(1998)- et al.
Action-perception patterns in virtual ball bouncing: combating system latency and tracking functional validity
J. Neurosci. Methods.
(2008) - et al.
The effects of display delay on simulator sickness
Displays
(2011) - et al.
The effects of 0.2 Hz varying latency with 20–100 ms varying amplitude on simulator sickness in a helmet mounted display
Displays
(2015) - et al.
Telepresence, time delay, and adaptation
Spat. Displays Spatial Instrum.
(1989) - et al.
A natural interface for remote operation of underwater robots
IEEE Comput. Graph.
(2017) - et al.
Effects of tracking technology, latency, and spatial jitter on object movement
- et al.
Effect of delay on dynamic targets tracking performance and behavior in virtual environment
- et al.
Reaching for objects in VR displays: lag and frame rate
ACM Trans. Comput. Interact.
(1994) - et al.
A timing model for vision-based control of industrial robot manipulators
IEEE Trans. Robot.
(2004)
Adaptation to visual feedback delays in a human manual tracking task
Exp. Brain Res.
On the genesis of abstract ideas
J. Exp. Psychol.
Measuring digital system latency from sensing to actuation at continuous 1-ms resolution
Presence
Frequency, not amplitude, of latency affects subjective sickness in a head-mounted display
Aerosp. Med. Hum. Perf.
Cited by (6)
3D face reconstruction and dense alignment with a new generated dataset
2021, DisplaysCitation Excerpt :3D face reconstruction and face alignment are two fundamental topics in computer vision, as they are essential preprocessing steps for many facial analysis tasks [1–4], such as recognition [5–10], animation [11,12], tracking [13–16], attribute classification [17], and image restoration [18–21]. It also can be applied to virtual reality [2,22], visual interactive [23,24] and display [25]. Most traditional studies [26,27] have focused on 3D Morphable Model (3DMM) parametric regression based on optimization algorithms and the iterative closest point [26].
Using head-mounted displays to examine adaptation and calibration under varying perturbations
2021, DisplaysCitation Excerpt :Based on findings of Wilson and colleagues [27] that task performance was lower under conditions of varying latency, it was hypothesized that there would be a main effect of latency, with those in the constant latency condition showing lower time-to-hit and higher accuracy than those in the varying latency condition. In Wilson and colleagues study [27], participants showed learning within the session, particularly in the varying latency condition. Therefore, it was hypothesized that there would be a main effect of session, such that in session three participants would show less time-to-hit and more accuracy than that of session one or two.
Effects of Constant and Time-Varying Display Lag on DVP and Cybersickness When Making Head-Movements in Virtual Reality
2023, International Journal of Human-Computer InteractionValidation of a novel bicycle simulator with realistic lateral and roll motion
2023, Vehicle System DynamicsA Kinematic Evaluation of Linear and Parabolic Pointing in Virtual Reality
2021, Proceedings of the Human Factors and Ergonomics SocietyAn immersive multi-user virtual reality for emergency simulation training: Usability study
2020, JMIR Serious Games
Michael L. Wilson is a research psychologist at the U.S. Army Aeromedical Research Laboratory. He received his PhD in human factors psychology from Clemson University in 2016.
Sarah C. Beadle is a graduate research assistant at Clemson University in human factors psychology. She received her MS in applied psychology at Clemson University in 2019.
Amelia J. Kinsella is a human factors engineer at Fort Hill Group, LLC. She received her PhD in human factors psychology from Clemson University in 2018.
Ryan S. Mattfeld is an assistant professor of computer science at Elon University. He received his PhD in electrical engineering from Clemson University in 2018.
Adam Hoover is a professor of electrical and computer engineering at Clemson University. He received his PhD in computer science and engineering from the University of South Florida in 1996.
Eric R. Muth is a professor of psychology at Clemson University. He received his PhD in psychology from The Pennsylvania State University in 1997.
- ☆
This paper was recommended for publication by Richard H.Y. So.