Reflectance transformation imaging (RTI) is a computational photography technique widely used in the cultural heritage and material science domains to characterize relieved surfaces. It basically consists of capturing multiple images from a fixed viewpoint with varying lights. Handling the potentially huge amount of information stored in an RTI acquisition that consists typically of 50–100 RGB values per pixel, allowing data exchange, interactive visualization, and material analysis, is not easy. The solution used in practical applications consists of creating “relightable images” by approximating the pixel information with a function of the light direction, encoded with a small number of parameters. This encoding allows the estimation of images relighted from novel, arbitrary lights, with a quality that, however, is not always satisfactory. In this paper, we present NeuralRTI, a framework for pixel-based encoding and relighting of RTI data. Using a simple autoencoder architecture, we show that it is possible to obtain a highly compressed representation that better preserves the original information and provides increased quality of virtual images relighted from novel directions, especially in the case of challenging glossy materials. We also address the problem of validating the relight quality on different surfaces, proposing a specific benchmark, SynthRTI, including image collections synthetically created with physical-based rendering and featuring objects with different materials and geometric complexity. On this dataset and as well on a collection of real acquisitions performed on heterogeneous surfaces, we demonstrate the advantages of the proposed relightable image encoding.

中文翻译：

---

神经反射变换成像

反射变换成像 (RTI) 是一种计算摄影技术，广泛用于文化遗产和材料科学领域，以表征浮雕表面。它基本上包括从固定视点用不同的光线捕捉多个图像。处理 RTI 采集中存储的潜在大量信息（通常由每像素 50-100 个 RGB 值组成），允许数据交换、交互式可视化和材料分析并不容易。实际应用中使用的解决方案包括通过用少量参数编码的光方向函数近似像素信息来创建“可重新照明的图像”。这种编码允许估计从新的、任意的灯光重新点亮的图像，但其质量并不总是令人满意。在本文中，我们介绍了 NeuralRTI，这是一个基于像素的 RTI 数据编码和重新照明框架。使用简单的自动编码器架构，我们表明可以获得高度压缩的表示，该表示可以更好地保留原始信息并提供从新方向重新点亮的虚拟图像的更高质量，特别是在具有挑战性的光泽材料的情况下。我们还解决了验证不同表面上的重新照明质量的问题，提出了一个特定的基准 SynthRTI，包括使用基于物理的渲染合成创建的图像集合，并以具有不同材料和几何复杂性的对象为特色。在这个数据集以及在异质表面上执行的真实采集的集合上，我们展示了所提出的可重新照明的图像编码的优势。