Person re-identification (Re-ID) is an important problem in video surveillance for matching pedestrian images across non-overlapping camera views. Currently, most works focus on RGB-based Re-ID. However, RGB images are not well suited to a dark environment; consequently, infrared (IR) imaging becomes necessary for indoor scenes with low lighting and 24-h outdoor scene surveillance systems. In such scenarios, matching needs to be performed between RGB images and IR images, which exhibit different visual characteristics; this cross-modality matching problem is more challenging than RGB-based Re-ID due to the lack of visible colour information in IR images. To address this challenge, we study the RGB-IR cross-modality Re-ID (RGB-IR Re-ID) problem. Rather than applying existing cross-modality matching models that operate under the assumption of identical data distributions between training and testing sets to handle the discrepancy between RGB and IR modalities for Re-ID, we cast learning shared knowledge for cross-modality matching as the problem of cross-modality similarity preservation. We exploit same-modality similarity as the constraint to guide the learning of cross-modality similarity along with the alleviation of modality-specific information, and finally propose a Focal Modality-Aware Similarity-Preserving Loss. To further assist the feature extractor in extracting shared knowledge, we design a modality-gated node as a universal representation of both modality-specific and shared structures for constructing a structure-learnable feature extractor called Modality-Gated Extractor. For validation, we construct a new multi-modality Re-ID dataset, called SYSU-MM01, to enable wider study of this problem. Extensive experiments on this SYSU-MM01 dataset show the effectiveness of our method. Download link of dataset: https://github.com/wuancong/SYSU-MM01 .

中文翻译：

---

通过跨模态相似性保存的RGB-IR人重识别

行人重新识别 (Re-ID) 是视频监控中的一个重要问题，用于在非重叠摄像机视图中匹配行人图像。目前，大多数工作都集中在基于 RGB 的 Re-ID。然而，RGB 图像不太适合黑暗环境；因此，对于光线不足的室内场景和 24 小时室外场景监控系统，红外 (IR) 成像变得必要。在这种场景下，需要对呈现不同视觉特征的RGB图像和IR图像进行匹配；由于红外图像中缺乏可见的颜色信息，这种跨模态匹配问题比基于 RGB 的 Re-ID 更具挑战性。为了应对这一挑战，我们研究了 RGB-IR 跨模态 Re-ID (RGB-IR Re-ID) 问题。我们没有应用在训练集和测试集之间数据分布相同的假设下运行的现有跨模态匹配模型来处理 Re-ID 的 RGB 和 IR 模态之间的差异，而是将跨模态匹配的学习共享知识作为问题跨模态相似性保存。我们利用相同模态相似性作为约束来指导跨模态相似性的学习以及模态特定信息的缓解，并最终提出了一种 Focal Modality-Aware Similarity-Preserving Loss。为了进一步帮助特征提取器提取共享知识，我们设计了一个模态门控节点作为模态特定结构和共享结构的通用表示，以构建一个称为模态门控提取器的结构可学习特征提取器。为了验证，我们构建了一个新的多模态 Re-ID 数据集，称为 SYSU-MM01，以便更广泛地研究这个问题。在这个 SYSU-MM01 数据集上的大量实验表明了我们方法的有效性。数据集下载链接：https://github.com/wuancong/SYSU-MM01。