In this paper, we propose spatiotemporal co-attention hybrid neural network (SC-HNN), a novel hybrid neural network model with both spatial and temporal attention mechanisms for pose-invariant inertial odometry. The main idea is to extract both local and global features from a window of IMU measurements for velocity prediction. SC-HNN leverages the convolutional neural network (CNN) to capture the sectional features and long short-term memory (LSTM) recurrent neural network (RNN) to extract the long-range dependencies. Attention mechanisms are designed and embedded in both CNN and LSTM modules for better model representation. Specifically, in the CNN attention block, the convolved features are refined along both channel and element dimensions. For the LSTM module, softmax scoring is applied to update the weights of the hidden states along the temporal axis. We evaluate SC-HNN on the benchmark with the largest and most natural IMU data, RoNIN. Extensive ablation experiments demonstrate the effectiveness of our SC-HNN model. Compared with the state of the art, the 50th percentile accuracy of SC-HNN is 18.21% higher and the 90th percentile accuracy is 21.15% higher for all the phone holders not appeared in the training set. The real scenario inertial tracking trials in the CUHK campus further prove the superior generalization ability of the SC-HNN model. Note to Practitioners—This paper aims at improving the localization accuracy of deep inertial odometry. We focus on the problem of indoor localization only from the low-cost IMU embedded in the smartphone without any restriction on the phone’s daily use. IMU is a perfect solution for indoor localization because of its low power consumption, high privacy protection, and external infrastructure free. This paper suggests a novel hybrid convolutional and recurrent neural network with a set of carefully designed attention mechanisms to improve the representation ability of deep inertial odometry model. Specifically, the convolutional layer is applied to extract the local spatial features among the 6D IMU signals, following a cascaded channel attention module and element attention module to boost the representation ability of CNN. The complex long-term dependencies are then identified by the LSTM layers. To adaptively capture the temporal features of the multimodal inertial signals, an attention mechanism is applied to weigh the hidden states for the generation of the final features. The effectiveness of the SC-HNN design is validated by extensive ablation studies. To the best of our knowledge, our model is the first HNN fused attention mechanism for inertial tracking. Extensive experiments show that the proposed method outperforms the state of the art.

中文翻译：

---

基于6D IMU的时空协同注意混合神经网络行人定位

在本文中，我们提出了时空协同注意混合神经网络 (SC-HNN)，这是一种具有空间和时间注意机制的新型混合神经网络模型，用于姿势不变的惯性里程计。主要思想是从 IMU 测量窗口中提取局部和全局特征以进行速度预测。SC-HNN 利用卷积神经网络 (CNN) 捕获截面特征，利用长短期记忆 (LSTM) 循环神经网络 (RNN) 提取长程依赖性。注意力机制被设计并嵌入到 CNN 和 LSTM 模块中，以实现更好的模型表示。具体来说，在 CNN 注意力块中，卷积特征在通道和元素维度上都得到了细化。对于 LSTM 模块，softmax 评分用于更新时间轴上隐藏状态的权重。我们在具有最大和最自然的 IMU 数据 RoNIN 的基准上评估 SC-HNN。广泛的消融实验证明了我们的 SC-HNN 模型的有效性。与现有技术相比，对于所有未出现在训练集中的电话持有者，SC-HNN 的第 50 个百分位精度提高了 18.21%，第 90 个百分位精度提高了 21.15%。中大校园真实场景惯性跟踪试验进一步证明了SC-HNN模型优越的泛化能力。从业者须知——本文旨在提高深度惯性里程计的定位精度。我们只关注智能手机中嵌入的低成本 IMU 的室内定位问题，对手机的日常使用没有任何限制。IMU低功耗、高隐私保护、无需外部基础设施，是室内定位的完美解决方案。本文提出了一种新颖的混合卷积和递归神经网络，具有一组精心设计的注意机制，以提高深度惯性里程计模型的表示能力。具体来说，卷积层用于提取 6D IMU 信号中的局部空间特征，然后级联通道注意模块和元素注意模块以提高 CNN 的表示能力。然后由 LSTM 层识别复杂的长期依赖关系。为了自适应地捕获多模态惯性信号的时间特征，应用注意力机制来权衡隐藏状态以生成最终特征。广泛的消融研究验证了 SC-HNN 设计的有效性。据我们所知，我们的模型是第一个用于惯性​​跟踪的 HNN 融合注意机制。大量实验表明，所提出的方法优于现有技术。