Visual place recognition (VPR) is the process of recognising a previously visited place using visual information, often under varying appearance conditions and viewpoint changes and with computational constraints. VPR is related to the concepts of localisation, loop closure, image retrieval and is a critical component of many autonomous navigation systems ranging from autonomous vehicles to drones and computer vision systems. While the concept of place recognition has been around for many years, VPR research has grown rapidly as a field over the past decade due to improving camera hardware and its potential for deep learning-based techniques, and has become a widely studied topic in both the computer vision and robotics communities. This growth however has led to fragmentation and a lack of standardisation in the field, especially concerning performance evaluation. Moreover, the notion of viewpoint and illumination invariance of VPR techniques has largely been assessed qualitatively and hence ambiguously in the past. In this paper, we address these gaps through a new comprehensive open-source framework for assessing the performance of VPR techniques, dubbed “VPR-Bench”. VPR-Bench (Open-sourced at: https://github.com/MubarizZaffar/VPR-Bench) introduces two much-needed capabilities for VPR researchers: firstly, it contains a benchmark of 12 fully-integrated datasets and 10 VPR techniques, and secondly, it integrates a comprehensive variation-quantified dataset for quantifying viewpoint and illumination invariance. We apply and analyse popular evaluation metrics for VPR from both the computer vision and robotics communities, and discuss how these different metrics complement and/or replace each other, depending upon the underlying applications and system requirements. Our analysis reveals that no universal SOTA VPR technique exists, since: (a) state-of-the-art (SOTA) performance is achieved by 8 out of the 10 techniques on at least one dataset, (b) SOTA technique in one community does not necessarily yield SOTA performance in the other given the differences in datasets and metrics. Furthermore, we identify key open challenges since: (c) all 10 techniques suffer greatly in perceptually-aliased and less-structured environments, (d) all techniques suffer from viewpoint variance where lateral change has less effect than 3D change, and (e) directional illumination change has more adverse effects on matching confidence than uniform illumination change. We also present detailed meta-analyses regarding the roles of varying ground-truths, platforms, application requirements and technique parameters. Finally, VPR-Bench provides a unified implementation to deploy these VPR techniques, metrics and datasets, and is extensible through templates.

视觉位置识别（VPR）是使用视觉信息来识别先前访问过的位置的过程，通常在变化的外观条件和视点变化下并具有计算约束。VPR与定位，回路闭合，图像检索的概念有关，并且是许多自动导航系统（从自动驾驶汽车到无人机和计算机视觉系统）的重要组成部分。尽管位置识别的概念已经存在了很多年，但由于摄像头硬件的改进及其在基于深度学习的技术中的潜力，VPR研究在过去的十年中迅速发展成为一个领域，并且在这两个领域都已成为广泛研究的主题。计算机视觉和机器人社区。但是，这种增长导致该领域的分散化和缺乏标准化，特别是关于绩效评估。此外，VPR技术的视点和照度不变性的概念在过去已被定性评估，因此在过去是模棱两可的。在本文中，我们通过一个名为“ VPR-Bench”的，用于评估VPR技术性能的新的，全面的开源框架来解决这些差距。VPR-Bench（开源：https：//github.com/MubarizZaffar/VPR-Bench）为VPR研究人员介绍了两项急需的功能：首先，它包含12个完全集成的数据集和10种VPR技术的基准，其次，它集成了一个全面的量化变量化数据集，用于量化视点和照度不变性。我们应用并分析了计算机视觉和机器人社区中针对VPR的流行评估指标，并讨论这些不同的指标如何根据基础应用程序和系统要求相互补充和/或替换。我们的分析表明，不存在通用的SOTA VPR技术，因为：（a）通过至少10个数据集上的10种技术中的8种实现了最先进的（SOTA）性能，（b）一个社区中的SOTA技术鉴于数据集和指标的差异，不一定会在其他方面产生SOTA性能。此外，由于以下原因，我们发现了关键的开放挑战：（c）所有10种技术在感知上混叠且结构化程度较低的环境中均遭受严重损失；（d）所有技术均受到视点方差的影响，其中横向更改的影响小于3D更改，并且（e）定向照明变化比均匀照明变化对匹配置信度的不利影响更大。我们还提供了有关变化的地面真相，平台，应用程序要求和技术参数的作用的详细荟萃分析。最后，VPR-Bench提供了一个统一的实施方案来部署这些VPR技术，指标和数据集，并且可以通过模板进行扩展。