Indoor way-finding method using IMU and magnetic tensor sensor measurements for visually impaired users

Abstract

This paper introduces a new indoor way-finding method for the visually impaired person (VIP) by utilizing the naturally-generated inertial and geomagnetic information. Reliable and accurate indoor orientation and localization are provided by newly designed sensor fusion algorithms, which take advantage of inertial and geomagnetic information and overcome the inherent problems of the naturally-generated signals, such as low signal-to-noise ratio (SNR) and high environmental sensitivity. Geomagnetic information compensates the sensor drift and accumulative error of the inertial sensors whereas the inertial sensors help to correct the orientation-related errors and drift of the magnetic fields. A parameter derived from the magnetic tensor is introduced for obstacle avoidance and object/destination approach, especially when the relatively large localization uncertainty exists. With a prototype developed based on the system design, several experiments under different indoor scenarios demonstrate that the proposed indoor-way finding method can guide the VIPs and avoid obstacles indoor efficiently and accurately.

Appendices

Appendix 1: Orientation representation with quaternion

Quaternions, which consist of four numbers as shown in Eq. (17), can uniquely determine a 3D rotation or the orientation of coordinate B relative to coordinate A.

$${\mathbf{q}}^{BA} = \left[ {\begin{array}{*{20}c} {q_{1} } & {q_{2} } & {q_{3} } & {q_{4} } \\ \end{array} } \right]$$

(17)

The inverse (denoted by subscript ⁻¹) or conjugate (denoted by subscript *) of the rotation quaternion represents the opposite rotation or swapped relative orientation, which is mathematically expressed in Eq. (18a, b).

$$\left( {{\mathbf{q}}^{BA} } \right)^{ - 1} = \left( {{\mathbf{q}}^{BA} } \right)^{*} = \left[ {\begin{array}{*{20}c} {q_{1} } & { - q_{2} } & { - q_{3} } & { - q_{4} } \\ \end{array} } \right] = {\mathbf{q}}^{AB}$$

(18a, b)

where q^AB represents the orientation of coordinate A with respect coordinate B. To represent a sequential orientation and coordinate transformation, the Hamilton product (denoted by $\otimes$) of the quaternion is introduced in Eq. (19).

$$\begin{aligned} {\mathbf{m}} \otimes {\mathbf{n}} & = \left[ {\begin{array}{*{20}c} {m_{1} } & {m_{2} } & {m_{3} } & {m_{4} } \\ \end{array} } \right] \otimes \left[ {\begin{array}{*{20}c} {n_{1} } & {n_{2} } & {n_{3} } & {n_{4} } \\ \end{array} } \right] \\ & = \left[ {\begin{array}{*{20}c} {m_{1} n_{1} - m_{2} n_{2} - m_{3} n_{3} - m_{4} n_{4} } \\ {m_{1} n_{2} + m_{2} n_{1} + m_{3} n_{4} - m_{4} n_{3} } \\ {m_{1} n_{3} - m_{2} n_{4} + m_{3} n_{1} + m_{4} n_{2} } \\ {m_{1} n_{4} + m_{2} n_{3} - m_{3} n_{2} + m_{4} n_{1} } \\ \end{array} } \right]^{{\text{T}}} \\ \end{aligned}$$

(19)

$${\mathbf{q}}^{CA} = {\mathbf{q}}^{CB} \otimes {\mathbf{q}}^{BA}$$

(20)

Assume another coordinate C is introduced and its orientation q^CB with respect to coordinate B is given. The orientation of C relative to A is represented with the quaternion product in Eq. (20).

Assume u^A is a vector described in coordinate A. A 0(zero) is inserted to this vector to make it a row vector containing 4 elements. Given the relative orientation of coordinate B represented with q^AB, the same vector described in coordination B is expressed in Eq. (21).

$${\mathbf{u}}^{B} = {\mathbf{q}}^{BA} \otimes {\mathbf{u}}^{A} \otimes \left( {{\mathbf{q}}^{BA} } \right)^{ * }$$

(21)

It can also be represented in a rotation matrix form.

$${\mathbf{u}}^{B} = {\mathbf{u}}^{A} \left[ {{\mathbf{R}}_{A}^{B} } \right]$$

(22a, b)

$$\left[ {{\mathbf{R}}_{A}^{B} } \right]{ = }\left[ {\begin{array}{*{20}c} {2q_{1}^{2} + 2q_{2}^{2} - 1} & {2\left( {q_{2} q_{3} + q_{1} q_{4} } \right)} & {2\left( {q_{2} q_{4} - q_{1} q_{3} } \right)} \\ {2\left( {q_{2} q_{3} - q_{1} q_{4} } \right)} & {2q_{1}^{2} + 2q_{3}^{2} - 1} & {2\left( {q_{3} q_{4} + q_{1} q_{2} } \right)} \\ {2\left( {q_{2} q_{4} + q_{1} q_{3} } \right)} & {2\left( {q_{3} q_{4} - q_{1} q_{2} } \right)} & {q_{1}^{2} + q_{4}^{2} - 1} \\ \end{array} } \right]$$

Appendix 2: MTS calibration

Assume the MTS is placed outside (no external magnetic anomalies), the measurements of the magnetic sensors in the MTS T consists of two components, MFD of the geomagnetic field B^G and internal magnetic anomalies B^A generated by the magnetic solder materials.

$${\mathbf{T}} = {\mathbf{B}}^{G} + {\mathbf{B}}^{A}$$

(22)

In order to estimate B^A, the MTS is placed outside with orientation shown in Fig. 

Fig. 18

MTS orientation used for B_A estimation

18. P2 and P3 can the positions when rotating 180° about z and y axis of the sensor frame. Assume internal magnetic anomalies ${\mathbf{B}}_{1}^{G} = \left[ {\begin{array}{*{20}c} {B_{x}^{G} } & {B_{y}^{G} } & {B_{z}^{G} } \\ \end{array} } \right]^{T}$ at P1, internal magnetic anomalies at P2 and P3 can be estimated with ${\mathbf{B}}_{2}^{G} = \left[ {\begin{array}{*{20}c} { - B_{x}^{G} } & { - B_{y}^{G} } & {B_{z}^{G} } \\ \end{array} } \right]^{T}$ and ${\mathbf{B}}_{3}^{G} = \left[ {\begin{array}{*{20}c} { - B_{x}^{G} } & {B_{y}^{G} } & { - B_{z}^{G} } \\ \end{array} } \right]^{T}$. Assume MFD measurements at P1, P2 and P3 are T₁, T₂ and T₃ respectively, after applying Eq. (22) for each position, the internal magnetic anomalies BA can be estimated with Eq. (23a, b, c, d).

$$\begin{gathered} {\mathbf{B}}^{A} = \left[ {\begin{array}{*{20}c} {B_{x}^{A} } & {B_{y}^{A} } & {B_{z}^{A} } \\ \end{array} } \right]^{T} ,{\text{where }}B_{x}^{A} = \left( {T_{1x} + T_{2y} } \right)/2 \hfill \\ B_{y}^{A} = \left( {T_{1y} + T_{2y} } \right)/2,B_{z}^{A} = \left( {T_{1z} + T_{3z} } \right)/2 \hfill \\ \end{gathered}$$

(23a, b, c, d)