ABSTRACT

A salient scene is an area within an image that contains visual elements that stand out from surrounding areas. They are important for distinguishing landmarks in first-person-view (FPV) applications and determining spatial relations in images. The relative spatial relation between salient scenes acts as a visual guide that is easily accepted and understood by users in FPV applications. However, current digitally navigable maps and location-based services fall short of providing information on visual spatial relations for users. This shortcoming has a critical influence on the popularity and innovation of FPV applications. This paper addresses the issue by proposing a method for detecting visually salient scene areas (SSAs) and deriving their relative spatial relationships from continuous panoramas. This method includes three critical steps. First, an SSA detection approach is introduced by fusing region-based saliency derived from super-pixel segmentation and the frequency-tuned saliency model. The method focuses on a segmented landmark area in a panorama. Secondly, a street-view-oriented SSA generation method is introduced by matching and merging the visual SSAs from continuous panoramas. Thirdly, a continuous geotagged panorama-based referencing approach is introduced to derive the relative spatial relationships of SSAs from continuous panoramas. This information includes the relative azimuth, elevation angle, and the relative distance. Experiment results show that the error for the SSA relative azimuth angle is approximately ± 6° (with an average error of 2.67°), and the SSA relative elevation angle is approximately ± 4° (with an average error of 1.32°) when using Baidu street-view panoramas. These results demonstrate the feasibility of the proposed approach. The method proposed in this study can facilitate the development of FPV applications such as augmented reality (AR) and pedestrian navigation using proper spatial relation.

突出场景是图像中包含从周围区域突出的视觉元素的区域。它们对于区分第一人称视角（FPV）应用程序中的界标和确定图像中的空间关系非常重要。显着场景之间的相对空间关系充当FPV应用程序中的用户容易接受和理解的视觉指南。但是，当前的数字可导航地图和基于位置的服务不能为用户提供有关视觉空间关系的信息。此缺点对FPV应用程序的普及和创新具有至关重要的影响。本文通过提出一种检测视觉上显着的场景区域（SSA）并从连续全景图推导它们的相对空间关系的方法来解决这个问题。该方法包括三个关键步骤。第一，通过融合从超像素分割得到的基于区域的显着性和频率调谐的显着性模型，引入了SSA检测方法。该方法着重于全景图中的分割地标区域。其次，通过匹配和合并来自连续全景图的视觉SSA，介绍了一种面向街景的SSA生成方法。第三，引入了基于连续地理标记全景图的参考方法，以从连续全景图中推导SSA的相对空间关系。该信息包括相对方位角，仰角和相对距离。实验结果表明，SSA相对方位角的误差约为±6°（平均误差为2.67°），SSA相对仰角约为±4°（平均误差为1。使用百度街景全景时（32°）。这些结果证明了该方法的可行性。本研究中提出的方法可以促进FPV应用程序的开发，例如使用适当的空间关系进行增强现实（AR）和行人导航。