Abstract. The fusion of depth acquired actively with the depth estimated passively proved its significance as an improvement strategy for gaining depth. This combination allows us to benefit from two sources of modalities such that they complement each other. To fuse two sensor data into a more accurate depth map, we must consider the limitations of active sensing such as low lateral resolution while combining it with a passive depth map. We present an approach for the fusion of active time-of-flight depth and passive stereo depth in an accurate way. We propose a multimodal sensor fusion strategy that is based on a weighted energy optimization problem. The weights are generated as a result of combining the edge information from a texture map and active and passive depth maps. The objective evaluation of our fusion algorithm shows an improved accuracy of the generated depth map in comparison with the depth map of every single modality and with the results of other fusion methods. Additionally, a visual comparison of our result shows a better recovery on the edges considering the wrong depth values estimated in passive stereo. Moreover, the left and right consistency check on the result illustrates the ability of our approach to consistently fuse sensors.

中文翻译：

---

飞行时间与立体深度融合的引导优化框架

摘要。主动获取的深度与被动估计的深度的融合证明了其作为获得深度的改进策略的重要性。这种组合使我们能够从两种模式来源中受益，使它们相互补充。为了将两个传感器数据融合成更准确的深度图，我们必须考虑主动传感的局限性，例如在将其与被动深度图结合时的低横向分辨率。我们提出了一种以准确方式融合主动飞行时间深度和被动立体深度的方法。我们提出了一种基于加权能量优化问题的多模态传感器融合策略。权重是结合来自纹理图和主动和被动深度图的边缘信息而生成的。我们的融合算法的客观评估表明，与每种模式的深度图和其他融合方法的结果相比，生成的深度图的准确性有所提高。此外，考虑到被动立体中估计的错误深度值，我们结果的视觉比较显示边缘恢复更好。此外，对结果的左右一致性检查说明了我们的方法能够始终如一地融合传感器。