Photometric stereo provides an important method for high-fidelity 3D reconstruction based on multiple intensity images captured under different illumination directions. In this paper, we present a complete framework, including a multilight source illumination and acquisition hardware system and a two-stage convolutional neural network (CNN) architecture, to construct inter- and intraframe representations for accurate normal estimation of non-Lambertian objects. We experimentally investigate numerous network design alternatives for identifying the optimal scheme to deploy inter- and intraframe feature extraction modules for the photometric stereo problem. Moreover, we propose utilizing the easily obtained object mask to eliminate adverse interference from invalid background regions in intraframe spatial convolutions, thus effectively improving the accuracy of normal estimation for surfaces made of dark materials or with cast shadows. Experimental results demonstrate that the proposed masked two-stage photometric stereo CNN model (MT-PS-CNN) performs favourably against state-of-the-art photometric stereo techniques in terms of both accuracy and efficiency. In addition, the proposed method is capable of predicting accurate and rich surface normal details for non-Lambertian objects of complex geometry and performs stably given inputs captured in both sparse and dense lighting distributions.

光度立体为基于在不同照明方向下捕获的多个强度图像进行高保真 3D 重建提供了重要方法。在本文中，我们提出了一个完整的框架，包括一个多光源照明和采集硬件系统和一个两级卷积神经网络 (CNN) 架构，以构建帧间和帧内表示，以准确估计非朗伯对象的法线。我们通过实验研究了许多网络设计替代方案，以确定为光度立体问题部署帧间和帧内特征提取模块的最佳方案。此外，我们建议利用容易获得的对象掩码来消除帧内空间卷积中无效背景区域的不利干扰，从而有效地提高了对由深色材料或投射阴影构成的表面的法线估计的准确性。实验结果表明，所提出的掩蔽两阶段光度立体 CNN 模型（MT-PS-CNN）在准确性和效率方面均优于最先进的光度立体技术。此外，所提出的方法能够预测复杂几何的非朗伯对象的准确和丰富的表面法线细节，并在稀疏和密集照明分布中捕获的给定输入稳定执行。实验结果表明，所提出的掩蔽两阶段光度立体 CNN 模型（MT-PS-CNN）在准确性和效率方面均优于最先进的光度立体技术。此外，所提出的方法能够为复杂几何的非朗伯对象预测准确和丰富的表面法线细节，并在稀疏和密集照明分布中捕获的给定输入稳定执行。实验结果表明，所提出的掩蔽两阶段光度立体 CNN 模型（MT-PS-CNN）在准确性和效率方面均优于最先进的光度立体技术。此外，所提出的方法能够预测复杂几何的非朗伯对象的准确和丰富的表面法线细节，并在稀疏和密集照明分布中捕获的给定输入稳定执行。