Unsupervised monocular visual odometry with decoupled camera pose estimation

Abstract

Drift or error accumulation is an inevitable challenge in visual odometry (VO). To alleviate this issue, most of learning-based VO methods focus on various long and short term sequential learning schemes, while lose sight of the fact that inaccurate rotation estimate is the main source of VO drift. They usually estimate the six degrees of freedom (DoFs) of the camera motion simultaneously, without considering the inherent rotation-translation ambiguity. In this paper, we start from the designs of a cascade decoupled structure and a residual-based decoupled pose refinement scheme for accurate pose estimation. Then we extend them to an unsupervised monocular VO framework, which estimates the 3D camera poses by decoupling the estimations of rotation, translation and scale. Our VO model consists of three components: a monocular depth estimation, a decoupled pose estimation and a decoupled pose refinement. The first component learns the metric scale and depth cues by using stereo pairs for training, and predicts the absolute depth of monocular inputs. The latter two separate the estimation and refinement of rotation and translation. To improve the robustness of the rotation estimation, we use the unit quaternion, instead of the Euler angles, to represent 3D rotation. We have evaluated our model on the KITTI Visual Odometry Evaluation benchmark. Comparison experiments demonstrate that our method is superior to the state-of-the-art unsupervised VO methods, and can achieve comparable results with the supervised ones.

Introduction

Visual odometry(VO) [1] recovers the trajectory of a moving platform from the image sequences captured by one or more cameras mounted on the moving platform. The key of VO is to estimate the 6-DOF camera pose relative to a fixed reference frame. As an essential task in computer vision, the applications of VO range from autonomous driving and robotics to augmented reality, etc. In the past few decades, it has drawn great attention of the researchers from both academia and industry.

Benefiting from the learning ability of convolutional neural networks (CNN), many CNN-based VO methods have been proposed, and made significant progress in recent years [2], [3], [4]. They trained CNN models in an end-to-end supervised / unsupervised fashion, which takes several consecutive frames as input, and outputs the relative camera motion between frames. Obviously, inaccurate motion estimation inevitably results in drift errors that accumulate over time. In order to alleviate the effect of VO drift, most of the existing VO networks focus on various long and short term sequential learning schemes. However, they lose sight of the fact that inaccurate rotation estimate is the main source of VO drift [5], [6]. They usually use a single fully convolutional network (FCN) to simultaneously estimate the rotation and translation of camera motion, without considering the inherent rotation-translation ambiguity.

In this paper, we first focus on designing a cascade decoupled structure for camera motion estimation and refinement, which formulates the camera motion estimation in unit quaternion and decouples the rotation and the translation estimation. We then extend this structure into a general unsupervised monocular VO framework. Note that we adopt an unsupervised paradigm instead of a supervised one for two reasons:

• Due to the lack of guidance and constraint from the ground truth pose information, unsupervised methods are more susceptible to the drift problem, that makes them far less accurate than the supervised VO methods.

• Large VO datasets with high quality annotations are often difficult and expensive to obtain. Unsupervised methods are less affected by the dataset limitation. They usually learn depth and camera motion jointly without any ground truth information.

To the best of our knowledge, only limited learning-based research has been conducted on the decoupled pose estimation [7], [8], [9]. Compared with the state-of-the-art unsupervised VO methods, the main contributions of our work are in two aspects:

• The design of a cascade decoupled structure for camera motion estimation. It effectively separates the rotation and translation estimation, which significantly improves the accuracy of rotation estimation, and then further promotes the accuracy of translation estimation. Additionally, we employ unit quaternions [10], [11] for the rotation representation to increase the robustness of rotation estimation.

• An unsupervised VO framework with decoupled camera pose estimation and refinement. It estimates the camera motion by decoupling estimations of rotation, translation and scale, and allows solving in cascade for these three transformations. Our framework consists of three components: a monocular depth estimation, a decoupled pose estimation and a residual-based decoupled pose refinement. We do not adopt any sequential learning schemes with the aim to show the effectiveness of our cascade decoupled structure clearly.

The evaluation experiments on the public KITTI Visual Odometry Evaluation benchmark [12] manifest the advantages and effectiveness of our paradigm. Our model outperforms the state-of-the-art unsupervised VO methods, and can achieve comparable results with the supervised ones.