Fiber optical multifunctional human-machine interface for motion capture, temperature, and contact force monitoring

https://doi.org/10.1016/j.optlaseng.2020.106018Get rights and content

Highlights

  • A compact and wireless human machine interface for motion capture, gesture recognition and medical therapy based on fiber optical sensor technology.

  • Multifunctional fiber optical sensor platform based on fiber Bragg grating sensors for bending, contact force, torsion, and temperature sensing within a standard telecommunication glass fiber.

  • A novel fiber optical rotation sensor based on so-called Edge-FBG, processed with femtosecond laser point-by-point writing.

Abstract

A compact and wireless multifunctional human-machine interface (HMI) based on fiber Bragg grating (FBG) technology is presented. The interface is embedded into a glove and enables the motion capture of a human hand with finger tracking and hand wrist rotation monitoring, as well as temperature and contact force detection. For the realization of a complex hand tracking, a novel fiber optical rotation sensor based on three-dimensional shape sensing with Edge-FBG is presented. The data are evaluated within a micro-interrogator and wireless transferred to a smart glass in real-time. The HMI addressed different applications in motion capture for virtual reality (VR) or augmented reality (AR).

Introduction

In the last several years, a great amount of work has already been done in human-machine interface (HMI) research to create user interfaces by directly employing natural communication and manipulation skills of humans [1]. The adaption of direct sensing in HMI could be applied in a wide range of fields like e.g. Virtual Reality (VR) or Augmented Reality (AR) for virtual engineering, games, or the remotely control located implements [2]. However, it requires the capturing of various human movements like the motion of head, face, hand, or even the whole body [3]. Among them, the human hand, which has more than 20 degree-of-freedoms, forms the most effective interaction tool due to its dexterous functionality in communication and manipulation [4]. Thus, the direct use of human hand as an input device is one of the most attractive method for natural HMI [1].

Currently, most approaches are based on computer visions and camera systems, respectively. However, this technology has several limitations as it may have particular requirements for the application, the system is operating in, or the ambient light condition. Additionally, complex calibration and setup procedures need to be done to obtain precise measurements. In particular, the accurate motion capture of a human hand with low latency is still a challenge due to the high-required computing power. Compared to a sensory data glove, this system has the potential to provide more convenient and accurate approaches with high flexibility. As a result, many approaches of data gloves have been realized and employed in the fields of e.g. robotic control and surgery tracking for medical applications [5], [6], [7]. The most commonly used sensors in data gloves are resistance, acoustic, and magnetic tracking sensors [8,9]. Those sensors are integrated in a glove and attached on the joints of fingers [6]. Nevertheless, for the complete motion and sensory capture of hand, many different types of sensors have to be embedded in the glove. This sensor fusion increases the complexity of a data glove system and also reduce its portability and application use.

Optical fiber sensors provide a potential alternative to these traditional electrical sensors because of its high accuracy, long-term stability, small size and weight, as well as the possibility to monitor several measurement quantities. Among the optical fiber sensors, the fiber Bragg grating (FBG) sensor present the entire set of advantages related to optical fiber sensors [10]. Thus the FBG sensors have been selected for the hand and finger motion capture in many research projects and various approaches have been followed by using FBG strain sensors [10], [11], [12], [13], [14]. Due to the high elongation level of the finger joint (>15%) [15], complex design of geometry are required for FBG strain sensors to avoid breakage of the glass fiber. Other possible approaches such as three-dimensional (3D) FBG shape sensors have also been applied in data glove applications and can achieve a complete motion capture of hand [16], [17], [18]. Because of the complex fabrication process of such 3D shape sensor, they are not be chosen for finger bending detection.

In this paper, a data glove is presented as a multifunctional sensor platform with finger tracking, contact force, temperature and hand wrist rotation sensor based only on FBG technology.

Section snippets

Setup of data glove

The HMI for the fiber optical data glove is built up modular and consists of fiber optical bending, rotation, contact force, and temperature sensors, as well as a self-developed miniaturized fiber optical readout unit with data evaluation. All sensors are produced with infrared femtosecond laser in 800 nm single-mode glass fiber (Fibercore, SM800) by the point-by-point (pbp) processing technology described in [19]. The fiber optical sensors with their sensory functions are presented in the

Structure of data glove

The design of the glove is showed in Fig. 11a. All fingers are connected with one optical fiber, in every finger one optical bending sensor is embedded. The bending of middle joint is measured by the sensor and the other two joints are calculated by multiplexing the angle from middle joint with a factor. At the end of the glass fiber strand the force and temperature sensor are embedded. The Y-form part of rotation sensor is fixed on the back of hand and the other side is fixed 10 cm behind on

Conclusion and summary

A novel approach of fiber optical multifunctional human-machine interface for motion capture, temperature, and contact force monitoring based on different FBG sensors in a single mode 800nm glass fiber has been presented. The possibility of using such a multifunctional sensor platform for sensory tasks such as gesture recognition or whole body motion capturing offers new possibilities in human-machine interface.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (33)

  • K.N. Tarchanidis et al.

    Data glove with a force sensor

    IEEE Trans Instrum Meas

    (2003)
  • C.S. Fahn et al.

    Development of a data glove with reducing sensors based on magnetic induction

    IEEE Trans Ind Electron

    (2005)
  • A.F. Da Silva et al.

    FBG sensing glove for monitoring hand posture

    IEEE Sens J

    (2011)
  • C.A. Perez-Ramirez et al.

    An architecture for measuring joint angles using a long period fiber grating-based sensor

    Sensors (Switzerland)

    (2014)
  • E. Fujiwara et al.

    Development of a glove-based optical fiber sensor for applications in human-robot interaction

  • E. Fujiwara et al.

    Optical fiber tactile sensor for user interfaces

    Proc. Int. Symp. Consum. Electron. ISCE

    (2016)
  • Cited by (0)

    View full text