Neural Radiance Fields (NeRF) have recently gained a surge of interest within the computer vision community for its power to synthesize photorealistic novel views of real-world scenes. One limitation of NeRF, however, is its requirement of accurate camera poses to learn the scene representations. In this paper, we propose Bundle-Adjusting Neural Radiance Fields (BARF) for training NeRF from imperfect (or even unknown) camera poses -- the joint problem of learning neural 3D representations and registering camera frames. We establish a theoretical connection to classical image alignment and show that coarse-to-fine registration is also applicable to NeRF. Furthermore, we show that na\"ively applying positional encoding in NeRF has a negative impact on registration with a synthesis-based objective. Experiments on synthetic and real-world data show that BARF can effectively optimize the neural scene representations and resolve large camera pose misalignment at the same time. This enables view synthesis and localization of video sequences from unknown camera poses, opening up new avenues for visual localization systems (e.g. SLAM) and potential applications for dense 3D mapping and reconstruction.

中文翻译：

---

BARF：束调整神经辐射场

神经辐射场（NeRF）最近在计算机视觉界引起了极大的兴趣，因为它具有合成真实世界场景的逼真的新颖视图的功能。但是，NeRF的局限性之一是需要精确的相机姿势来学习场景表示。在本文中，我们提出了捆绑调整神经辐射场（BARF），用于从不完美（甚至未知）的相机姿势中训练NeRF-学习神经3D表示并注册相机框架的共同问题。我们建立了与经典图像对齐的理论联系，并表明从粗到精配准也适用于NeRF。此外，我们表明，在NeRF中天真地应用位置编码会对基于合成目标的配准产生负面影响。对合成数据和现实世界数据进行的实验表明，BARF可以有效地优化神经场景表示并同时解决大型相机姿态未对准的问题。这使得视图合成和来自未知相机姿势的视频序列的定位成为可能，从而为可视化定位系统（例如SLAM）开辟了新途径，并为密集的3D映射和重建提供了潜在的应用。