Spatial reasoning is an important component of human intelligence. We can imagine the shapes of 3D objects and reason about their spatial relations by merely looking at their three-view line drawings in 2D, with different levels of competence. Can deep networks be trained to perform spatial reasoning tasks? How can we measure their "spatial intelligence"? To answer these questions, we present the SPARE3D dataset. Based on cognitive science and psychometrics, SPARE3D contains three types of 2D-3D reasoning tasks on view consistency, camera pose, and shape generation, with increasing difficulty. We then design a method to automatically generate a large number of challenging questions with ground truth answers for each task. They are used to provide supervision for training our baseline models using state-of-the-art architectures like ResNet. Our experiments show that although convolutional networks have achieved superhuman performance in many visual learning tasks, their spatial reasoning performance on SPARE3D tasks is either lower than average human performance or even close to random guesses. We hope SPARE3D can stimulate new problem formulations and network designs for spatial reasoning to empower intelligent robots to operate effectively in the 3D world via 2D sensors. The dataset and code are available at https://ai4ce.github.io/SPARE3D.

中文翻译：

---

SPARE3D：三视图线图空间推理的数据集

空间推理是人类智能的重要组成部分。我们可以想象 3D 对象的形状并推理它们的空间关系，只需在 2D 中查看它们的三视图线图，具有不同的能力水平。可以训练深度网络来执行空间推理任务吗？我们如何衡量他们的“空间智能”？为了回答这些问题，我们提出了 SPARE3D 数据集。SPARE3D 基于认知科学和心理测量学，包含视图一致性、相机姿势和形状生成三类 2D-3D 推理任务，难度越来越大。然后我们设计了一种方法来自动生成大量具有挑战性的问题，并为每个任务提供真实答案。它们用于为使用 ResNet 等最先进架构训练我们的基线模型提供监督。我们的实验表明，尽管卷积网络在许多视觉学习任务中取得了超人的表现，但它们在 SPARE3D 任务上的空间推理性能要么低于人类的平均水平，要么甚至接近于随机猜测。我们希望 SPARE3D 可以激发空间推理的新问题公式和网络设计，使智能机器人能够通过 2D 传感器在 3D 世界中有效运行。数据集和代码可在 https://ai4ce.github.io/SPARE3D 获得。他们在 SPARE3D 任务上的空间推理性能要么低于人类的平均水平，要么甚至接近于随机猜测。我们希望 SPARE3D 可以激发空间推理的新问题公式和网络设计，使智能机器人能够通过 2D 传感器在 3D 世界中有效运行。数据集和代码可在 https://ai4ce.github.io/SPARE3D 获得。他们在 SPARE3D 任务上的空间推理性能要么低于人类的平均水平，要么甚至接近于随机猜测。我们希望 SPARE3D 可以激发空间推理的新问题公式和网络设计，使智能机器人能够通过 2D 传感器在 3D 世界中有效运行。数据集和代码可在 https://ai4ce.github.io/SPARE3D 获得。