Elsevier

Displays

Volume 61, January 2020, 101930
Displays

Task performance in a head-mounted display: The impacts of varying latency

https://doi.org/10.1016/j.displa.2019.101930Get rights and content

Highlights

  • Participants had lower accuracy and longer time-to-hit targets with varying latency.

  • Sickness was lower than previous similar conditions; the task may be a distractor.

  • Work is needed on relationships between varying latency, performance, and sickness.

Abstract

The purpose of this study was to determine how latency in a head-mounted display (HMD) affects human performance. Virtual environments (VEs) are used frequently for training. However, VEs can cause simulator sickness. Prior work in our laboratory has examined the role of varying latency in simulator sickness. However, the effect of varying latency on task performance has not been examined. Subjects participated in a repeated measures study where they were exposed to two different latency conditions in an HMD: constant (70 ms) and varying (70–270 ms). During each HMD exposure, subjects used a laser pointer to repeatedly “shoot” at laser targets while accuracy and time-to-hit were recorded. Subjects scored fewer hits and took longer to hit targets in the varying latency condition. These findings indicate that individuals exposed to varying latency perform worse than individuals exposed to a lower constant latency.

Introduction

Head-Mounted Displays (HMDs) are worn on the head and close to the eyes, providing users with an image resulting in a virtual visual surround. Current technology tracks head motion to update displays to the users’ point of view in the virtual surround, as they make head movements [1]. Although commonly used for entertainment, they are increasingly being used by workers performing remote operations, maintenance, engineering, and simulations [2], [3]. HMD users routinely encounter anomalies such as delays in feedback, or occlusion of peripheral vision, which interfere with their visual perception.

System latency is the time it takes for a real-world event to be sensed, processed, and displayed to the user. Latency is commonly in the range of tens to hundreds of milliseconds (ms) and causes control problems for users. In virtual reality systems, latency has been shown to confound tasks where timing is critical to successful task completion, such as pointing and object motion tasks [4], catching tasks [5], and ball bouncing tasks [6]. In robotics, latency has an impact on teleoperation and remote manipulation because of visual display limitations and delayed feedback [7], [8]. Delayed feedback can slow response time of the virtual display user, thereby decreasing the user’s ability to perform tracking and pursuit tasks [9], [10]. Feedback delays create a conflict because users cannot reliably use feedback from their actions to correct their behavior. While the perceptual sensitivity to latency has been investigated (e.g., [11]), there is little research evidence to support a critical threshold at which latency no longer produces negative side effects in terms of simulator sickness and decrements in performance.

System latency in HMDs is normally reported as a constant value. However, research has shown that system latency varies, fluctuating in both rate and magnitude [12]. In inertial based tracking systems, head-tracking sensor errors lead to varying latency in visual displays [12], that is, as a head-mounted display (HMD) wearer moves their head, the visual images being displayed are delayed in varying amounts, depending on the moment-to-moment accuracy of the head tracking. It has been shown that varying latency is related to simulator sickness [13], [14]. However, there is no research, to our knowledge, investigating the impact of variability in latency on performance (calibration) in an HMD.

The purpose of this study was to determine how varying latency in a head-mounted display affects human performance. The hypotheses were divided into two areas: accuracy and time-to-hit of targets. We expected performance to be better in the constant latency condition because individuals are known to be capable of calibrating to constant perceptual perturbations. However, when perturbations vary, individuals cannot calibrate and therefore performance will be more affected.

Section snippets

Participants

Thirty participants were recruited from the student, staff and faculty population of Clemson University via flyers and the Department of Psychology human subject pool website. Where relevant, some participants were given course credit. All participants were paid $15 for their first session and $35 for their second session. To be eligible, participants could not have any history of brain, heart, stomach, eye (other than contacts), or inner ear problems, or be pregnant. This research complied

Results

Of the 30 (15 male) participants in the study, one female participant did not return for the second experimental session and therefore her data were omitted from analyses. The demographics of the remaining 29 participants are shown in Table 3.1. There were no differences noted between the male and female groups for age, race or handedness.

Discussion

The purpose of this study was to determine how varying latency in a head-mounted display affects human performance. More specifically, the study examined the effects of system latency on accuracy and time needed to score a hit. Overall, the findings indicate that human performance is lower in the presence of varying latency.

A main effect of condition on accuracy was observed. Participants were less accurate in the varying latency condition than in the constant latency condition. This supports

Funding

This work was supported by the Clemson University Human Factors Institute for subject payments and supplies.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Michael L. Wilson is a research psychologist at the U.S. Army Aeromedical Research Laboratory. He received his PhD in human factors psychology from Clemson University in 2016.

References (28)

  • A.J.M. Foulkes et al.

    Adaptation to visual feedback delays in a human manual tracking task

    Exp. Brain Res.

    (2000)
  • M.I. Posner et al.

    On the genesis of abstract ideas

    J. Exp. Psychol.

    (1968)
  • W. Wu et al.

    Measuring digital system latency from sensing to actuation at continuous 1-ms resolution

    Presence

    (2013)
  • A. Kinsella et al.

    Frequency, not amplitude, of latency affects subjective sickness in a head-mounted display

    Aerosp. Med. Hum. Perf.

    (2016)
  • Cited by (6)

    • 3D face reconstruction and dense alignment with a new generated dataset

      2021, Displays
      Citation Excerpt :

      3D face reconstruction and face alignment are two fundamental topics in computer vision, as they are essential preprocessing steps for many facial analysis tasks [1–4], such as recognition [5–10], animation [11,12], tracking [13–16], attribute classification [17], and image restoration [18–21]. It also can be applied to virtual reality [2,22], visual interactive [23,24] and display [25]. Most traditional studies [26,27] have focused on 3D Morphable Model (3DMM) parametric regression based on optimization algorithms and the iterative closest point [26].

    • Using head-mounted displays to examine adaptation and calibration under varying perturbations

      2021, Displays
      Citation Excerpt :

      Based on findings of Wilson and colleagues [27] that task performance was lower under conditions of varying latency, it was hypothesized that there would be a main effect of latency, with those in the constant latency condition showing lower time-to-hit and higher accuracy than those in the varying latency condition. In Wilson and colleagues study [27], participants showed learning within the session, particularly in the varying latency condition. Therefore, it was hypothesized that there would be a main effect of session, such that in session three participants would show less time-to-hit and more accuracy than that of session one or two.

    • A Kinematic Evaluation of Linear and Parabolic Pointing in Virtual Reality

      2021, Proceedings of the Human Factors and Ergonomics Society

    Michael L. Wilson is a research psychologist at the U.S. Army Aeromedical Research Laboratory. He received his PhD in human factors psychology from Clemson University in 2016.

    Sarah C. Beadle is a graduate research assistant at Clemson University in human factors psychology. She received her MS in applied psychology at Clemson University in 2019.

    Amelia J. Kinsella is a human factors engineer at Fort Hill Group, LLC. She received her PhD in human factors psychology from Clemson University in 2018.

    Ryan S. Mattfeld is an assistant professor of computer science at Elon University. He received his PhD in electrical engineering from Clemson University in 2018.

    Adam Hoover is a professor of electrical and computer engineering at Clemson University. He received his PhD in computer science and engineering from the University of South Florida in 1996.

    Eric R. Muth is a professor of psychology at Clemson University. He received his PhD in psychology from The Pennsylvania State University in 1997.

    This paper was recommended for publication by Richard H.Y. So.

    View full text