The input data of photometric stereo have three dimensions: one photometric dimension of different lights and two spatial dimensions, i.e., rows and columns in the image coordinate. Recent deep-learning-based photometric stereo algorithms usually use 2-D/3-D convolutions to process the input with three dimensions. Assumptions that violate the natural characters of photometric stereo problem, e.g., spatial pixel interdependence and light permutation invariance, have to be made due to the dimension mismatch. In this article, we propose a self-attention photometric stereo network (SPS-Net), which can exploit the information in all three dimensions without violating these natural characters. In SPS-Net, the spatial information is extracted by convolutional layers and the photometric information is aggregated by the proposed photometric fusion blocks based on the self-attention mechanism. Extensive experiments on both synthetic and real-world data sets are conducted. The proposed SPS-Net achieved higher performance than the state-of-the-art algorithms photometric stereo task with dense lightings. Without any changes, the proposed algorithm also outperformed the benchmarks in sparse and light-information-robust photometric stereo tasks.

中文翻译：

---

SPS-Net：自注意力光度立体网络


光度立体的输入数据具有三个维度：一个不同光线的光度维度和两个空间维度，即图像坐标中的行和列。最近基于深度学习的光度立体算法通常使用 2-D/3-D 卷积来处理三维输入。由于尺寸不匹配，必须做出违反光度立体问题的自然特征的假设，例如空间像素相互依赖性和光排列不变性。在本文中，我们提出了一种自注意力光度立体网络（SPS-Net），它可以在不违反这些自然特征的情况下利用所有三个维度的信息。在SPS-Net中，通过卷积层提取空间信息，并通过基于自注意力机制的光度融合块聚合光度信息。对合成数据集和真实数据集进行了广泛的实验。所提出的 SPS-Net 比具有密集照明的最先进算法光度立体任务实现了更高的性能。在没有任何改变的情况下，所提出的算法在稀疏和光信息鲁棒的光度立体任务中也优于基准。