ABSTRACT In this paper, we propose a self-supervised optical flow-based approach to learn the rotation of an arbitrarily moving spherical camera. Nowadays, deep learning has enabled efficient learning of camera rotation efficiently. However, most approaches are fully supervised and require large datasets with ground-truth labels of the rotation, and these labels are difficult to acquire. We attempt to solve this problem by using a derotation operation of the spherical optical flow on a unit sphere. This operation decouples the camera rotation from the mixture of translational and rotational components, removing the effect of 3D information for rotation estimation. Therefore, we integrate a derotation layer into a convolutional neural network for regressing the camera rotation. This layer can be adopted for only spherical cameras, which can capture all-round information, and thus enables the network to be learned the camera rotation without using labeled training datasets. We experimentally demonstrate that our approach achieves the comparable performance for the rotation estimation to that of a fully supervised approach and that it outperforms a previously proposed approach. Moreover, transfer learning is conducted in new environments to confirm the benefit of the self-supervised learning. GRAPHICAL ABSTRACT

中文翻译：

---

用于球形相机旋转估计的自监督光流解旋网络

摘要在本文中，我们提出了一种基于自监督光流的方法来学习任意移动的球形相机的旋转。如今，深度学习使高效学习相机旋转成为可能。然而，大多数方法都是完全监督的，需要带有旋转的真实标签的大型数据集，而这些标签很难获得。我们试图通过在单位球面上使用球面光流的反旋转操作来解决这个问题。此操作将相机旋转与平移和旋转分量的混合解耦，从而消除了 3D 信息对旋转估计的影响。因此，我们将反旋转层集成到卷积神经网络中，用于回归相机旋转。该层只能用于球形相机，它可以捕获全方位的信息，从而使网络能够在不使用标记的训练数据集的情况下学习相机旋转。我们通过实验证明，我们的方法在旋转估计方面实现了与完全监督方法相当的性能，并且优于之前提出的方法。此外，迁移学习是在新环境中进行的，以确认自监督学习的好处。图形概要 迁移学习是在新环境中进行的，以确认自监督学习的好处。图形概要 迁移学习是在新环境中进行的，以确认自监督学习的好处。图形概要