Single-image 3-D reconstruction has long been a challenging problem. Recent deep learning approaches have been introduced to this 3-D area, but the ability to generate point clouds still remains limited due to inefficient and expensive 3-D representations, the dependency between the output and the number of model parameters, or the lack of a suitable computing operation. In this article, we present a novel deep-learning-based method to reconstruct a point cloud of an object from a single still image. The proposed method can be decomposed into two steps: feature fusion and deformation. The first step extracts both global and point-specific shape features from a 2-D object image, and then injects them into a randomly generated point cloud. In the second step, which is deformation, we introduce a new layer termed as GraphX that considers the interrelationship between points like common graph convolutions but operates on unordered sets. The framework can be applicable to realistic image data with background as we optionally learn a mask branch to segment objects from input images. To complement the quality of point clouds, we further propose an objective function to control the point uniformity. In addition, we introduce different variants of GraphX that cover from best performance to best memory budget. Moreover, the proposed model can generate an arbitrary-sized point cloud, which is the first deep method to do so. Extensive experiments demonstrate that we outperform the existing models and set a new height for different performance metrics in single-image 3-D reconstruction.

中文翻译：

---

单图像 3D 重建：重新思考点云变形


单图像 3D 重建长期以来一直是一个具有挑战性的问题。最近的深度学习方法已被引入到这个 3D 领域，但由于 3D 表示效率低下且昂贵、输出与模型参数数量之间的依赖性或缺乏合适的计算操作。在本文中，我们提出了一种基于深度学习的新颖方法，用于从单个静止图像重​​建对象的点云。该方法可以分解为两个步骤：特征融合和变形。第一步从二维对象图像中提取全局和特定点的形状特征，然后将它们注入到随机生成的点云中。在第二步（即变形）中，我们引入了一个名为 GraphX 的新层，它考虑点之间的相互关系，如常见的图卷积，但在无序集上运行。该框架可以适用于具有背景的真实图像数据，因为我们可以选择学习掩模分支来从输入图像中分割对象。为了补充点云的质量，我们进一步提出了一个目标函数来控制点均匀性。此外，我们还介绍了 GraphX 的不同变体，涵盖从最佳性能到最佳内存预算。此外，所提出的模型可以生成任意大小的点云，这是第一个这样做的深度方法。大量实验表明，我们优于现有模型，并为单图像 3D 重建中的不同性能指标设定了新的高度。