This paper proposes a novel fusion of an inertial measurement unit (IMU) and stereo camera method based on direct sparse odometry (DSO) and stereo DSO. It jointly optimizes all model parameters within a sliding window, including the inverse depth of all selected pixels and the internal or external camera parameters of all keyframes. The vision part uses a photometric error function that optimizes 3D geometry and camera pose in a combined energy functional. The proposed algorithm uses image blocks to extract neighboring image features and directly forms measurement residuals in the image intensity space. A fixed-baseline stereo camera solves scale drift. IMU information is accumulated between several frames using manifold pre-integration and is inserted into the optimization as additional constraints between keyframes. The scale and gravity inserted are incorporated into the stereo visual inertial odometry model and are optimized together with other variables such as poses. The experimental results show that the tracking accuracy and robustness of the proposed method are superior to those of the state-of-the-art fused IMU method. In addition, compared with previous semi-dense direct methods, the proposed method displays a higher reconstruction density and scene recovery.

中文翻译：

---

基于直接稀疏立体视觉惯性里程法的联合优化

本文提出了一种基于直接稀疏测距法（DSO）和立体DSO的惯性测量单元（IMU）和立体摄像机方法的新型融合方法。它共同优化了滑动窗口内的所有模型参数，包括所有选定像素的反深度以及所有关键帧的内部或外部摄像机参数。视觉部件使用光度误差函数，通过组合的能量函数优化3D几何形状和相机姿态。该算法利用图像块提取邻近图像特征，并直接在图像强度空间中形成测量残差。固定基线的立体摄像机可解决标度漂移。IMU信息使用流形预集成在几个帧之间累积，并作为关键帧之间的附加约束插入优化中。插入的比例和重力已合并到立体视觉惯性里程计模型中，并与其他变量（例如姿势）一起进行了优化。实验结果表明，该方法的跟踪精度和鲁棒性优于最新的融合IMU方法。另外，与以前的半密集直接方法相比，该方法具有更高的重建密度和场景恢复能力。