We propose an original approach to absolute pose and structure-from-motion (SfM) which handles rolling shutter (RS) effects. Unlike most existing methods which either augment global shutter projection with velocity parameters or impose continuous time and motion through pose interpolation, we use local differential constraints. These are established by drawing analogies with non-rigid 3D vision techniques, namely shape-from-template and non-rigid SfM (NRSfM). The proposed idea is to interpret the images of a rigid surface acquired by a moving RS camera as those of a virtually deformed surface taken by a GS camera. These virtually deformed surfaces are first recovered by relaxing the RS constraint using SfT or NRSfM. Then we upgrade the virtually deformed surface to the actual rigid structure and compute the camera pose and ego-motion by reintroducing the RS constraint. This uses a new 3D-3D registration procedure that minimizes a cost function based on the Euclidean 3D point distance. This is more stable and physically meaningful than the reprojection error or the algebraic distance used in previous work. Experimental results obtained with synthetic and real data show that the proposed methods outperform existing ones in terms of accuracy and stability, even in the known critical configurations.

中文翻译：

---

使用非刚性类比解决滚动快门 3D 视觉问题

我们提出了一种处理滚动快门 (RS) 效果的绝对姿势和运动结构 (SfM) 的原始方法。与使用速度参数增强全局快门投影或通过姿势插值施加连续时间和运动的大多数现有方法不同，我们使用局部微分约束。这些是通过绘制非刚性 3D 视觉技术的类比来建立的，即来自模板的形状和非刚性 SfM (NRSfM)。提出的想法是将移动 RS 相机获取的刚性表面的图像解释为 GS 相机拍摄的虚拟变形表面的图像。这些虚拟变形的表面首先通过使用 SfT 或 NRSfM 放松 RS 约束来恢复。然后我们将虚拟变形的表面升级为实际的刚性结构，并通过重新引入 RS 约束来计算相机姿势和自我运动。这使用了新的 3D-3D 配准程序，该程序基于欧几里得 3D 点距离最小化成本函数。这比以前工作中使用的重投影误差或代数距离更稳定，更具有物理意义。使用合成和真实数据获得的实验结果表明，即使在已知的关键配置中，所提出的方法在准确性和稳定性方面也优于现有方法。