This paper presents a hardware-adaptive feature modeling framework to automatically generate and optimize deep neural networks to support real-time feature extraction and matching on a given hardware platform. This framework consists of a deep feature extraction and matching pipeline and a neural architecture search scheme, with which deep neural networks can be automatically generated and optimized according to given hardware to achieve reliable real-time feature matching. Built on our feature matching approach, we also developed a real-time 3D scene reconstruction pipeline that could run adaptively on hardware with different computational performance. We designed experiments to validate the proposed matching and reconstruction pipelines on hardware platforms with different performance. The results demonstrated our algorithm’s effectiveness on both matching and reconstruction tasks.

本文提出了一种硬件自适应特征建模框架，该框架可自动生成和优化深度神经网络，以支持给定硬件平台上的实时特征提取和匹配。该框架由深度特征提取和匹配管道以及神经体系结构搜索方案组成，利用该方案可以根据给定的硬件自动生成和优化深度神经网络，以实现可靠的实时特征匹配。基于我们的特征匹配方法，我们还开发了实时3D场景重建管道，可以在具有不同计算性能的硬件上自适应运行。我们设计了实验，以验证具有不同性能的硬件平台上建议的匹配和重构管道。