Real-world data processing problems often involve various image modalities associated with a certain scene, including RGB images, infrared images, or multispectral images. The fact that different image modalities often share certain attributes, such as edges, textures, and other structure primitives, represents an opportunity to enhance various image processing tasks. This paper proposes a new approach to construct a high-resolution (HR) version of a low-resolution (LR) image, given another HR image modality as guidance, based on joint sparse representations induced by coupled dictionaries. The proposed approach captures complex dependency correlations, including similarities and disparities, between different image modalities in a learned sparse feature domain in lieu of the original image domain. It consists of two phases: coupled dictionary learning phase and coupled super-resolution phase. The learning phase learns a set of dictionaries from the training dataset to couple different image modalities together in the sparse feature domain. In turn, the super-resolution phase leverages such dictionaries to construct an HR version of the LR target image with another related image modality for guidance. In the advanced version of our approach, multistage strategy and neighbourhood regression concept are introduced to further improve the model capacity and performance. Extensive guided image super-resolution experiments on real multimodal images demonstrate that the proposed approach admits distinctive advantages with respect to the state-of-the-art approaches, for example, overcoming the texture copying artifacts commonly resulting from inconsistency between the guidance and target images. Of particular relevance, the proposed model demonstrates much better robustness than competing deep models in a range of noisy scenarios.

中文翻译：

---

通过耦合字典引起的联合稀疏表示的多模态图像超分辨率

现实世界的数据处理问题通常涉及与特定场景相关的各种图像模态，包括 RGB 图像、红外图像或多光谱图像。不同的图像模态通常共享某些属性，例如边缘、纹理和其他结构基元，这一事实代表了增强各种图像处理任务的机会。本文提出了一种新方法来构建低分辨率 (LR) 图像的高分辨率 (HR) 版本，该方法基于耦合字典诱导的联合稀疏表示，以另一种 HR 图像模态为指导。所提出的方法在代替原始图像域的学习稀疏特征域中捕获不同图像模态之间的复杂依赖相关性，包括相似性和差异性。它由两个阶段组成：耦合字典学习阶段和耦合超分辨率阶段。学习阶段从训练数据集中学习一组字典，以在稀疏特征域中将不同的图像模态耦合在一起。反过来，超分辨率阶段利用这样的字典来构建 LR 目标图像的 HR 版本，并使用另一种相关的图像模态进行指导。在我们方法的高级版本中，引入了多阶段策略和邻域回归概念，以进一步提高模型容量和性能。对真实多模态图像进行的大量引导图像超分辨率实验表明，所提出的方法相对于最先进的方法具有独特的优势，例如，克服通常由引导和目标图像之间的不一致导致的纹理复制伪影。特别相关的是，所提出的模型在一系列嘈杂场景中表现出比竞争深度模型更好的鲁棒性。