A full 360$^\circ$ × 180$^\circ$ image provides an unlimited field of view (FOV) and an immersive experience for the users without any loss of information of the surrounding. In this study, a deep learning based approach is proposed to synthesize a 360$^\circ$ image from a sparse set of images captured with a limited FOV. The proposed network consists of a cascade of the FOV estimation network and the panorama synthesis network. We propose a hierarchical generative network to synthesize high quality 360$^\circ$ panorama images. The design of progressive multi-scale generator and multiple discriminator reduces the high frequency artifact which is commonly observed in image synthesis using generative networks. The network is further compressed to run on a low-power device such as a smartphone using our proposed size and latency optimization. Experimental result demonstrates that the proposed method produces 360$^\circ$ panorama with satisfactory image quality of up to 512 × 1024 resolution. It is also shown that the proposed method outperforms the alternative method and can be generalized for non-panoramic scenes and real images captured by a smartphone camera.

中文翻译：

---

从低功耗设备上的稀疏图像集进行 360 度全景合成

一个完整的 360$^\circ$ × 180$^\circ$图像为用户提供了无限的视野（FOV）和身临其境的体验，而不会丢失任何周围信息。在这项研究中，提出了一种基于深度学习的方法来合成 360$^\circ$从使用有限 FOV 捕获的一组稀疏图像中提取的图像。所提出的网络由 FOV 估计网络和全景合成网络的级联组成。我们提出了一个分层生成网络来合成高质量的 360$^\circ$全景图像。渐进式多尺度生成器和多鉴别器的设计减少了在使用生成网络的图像合成中常见的高频伪影。使用我们建议的大小和延迟优化，网络被进一步压缩以在低功耗设备上运行，例如智能手机。实验结果表明，该方法产生了 360$^\circ$全景图像质量令人满意，分辨率高达 512 × 1024。还表明，所提出的方法优于替代方法，并且可以推广到非全景场景和智能手机相机拍摄的真实图像。