Point clouds-based 3D human pose estimation that aims to recover the 3D locations of human skeleton joints plays an important role in many AR/VR applications. The success of existing methods is generally built upon large scale data annotated with 3D human joints. However, it is a labor-intensive and error-prone process to annotate 3D human joints from input depth images or point clouds, due to the self-occlusion between body parts as well as the tedious annotation process on 3D point clouds. Meanwhile, it is easier to construct human pose datasets with 2D human joint annotations on depth images. To address this problem, we present a weakly supervised adversarial learning framework for 3D human pose estimation from point clouds. Compared to existing 3D human pose estimation methods from depth images or point clouds, we exploit both the weakly supervised data with only annotations of 2D human joints and fully supervised data with annotations of 3D human joints. In order to relieve the human pose ambiguity due to weak supervision, we adopt adversarial learning to ensure the recovered human pose is valid. Instead of using either 2D or 3D representations of depth images in previous methods, we exploit both point clouds and the input depth image. We adopt 2D CNN to extract 2D human joints from the input depth image, 2D human joints aid us in obtaining the initial 3D human joints and selecting effective sampling points that could reduce the computation cost of 3D human pose regression using point clouds network. The used point clouds network can narrow down the domain gap between the network input i.e. point clouds and 3D joints. Thanks to weakly supervised adversarial learning framework, our method can achieve accurate 3D human pose from point clouds. Experiments on the ITOP dataset and EVAL dataset demonstrate that our method can achieve state-of-the-art performance efficiently.

中文翻译：

---

来自点云的3D人体姿势估计的弱监督对抗学习。

基于点云的3D人体姿势估计旨在恢复人体骨骼关节的3D位置，在许多AR / VR应用中都发挥着重要作用。现有方法的成功通常建立在以3D人体关节注释的大规模数据上。但是，由于身体部位之间的自闭塞以及3D点云上的乏味注释过程，从输入深度图像或点云中注释3D人体关节是费力且容易出错的过程。同时，在深度图像上使用2D人体关节注释构建人体姿势数据集更加容易。为了解决这个问题，我们提出了一种基于点云的3D人体姿态估计的弱监督对抗学习框架。与现有的深度图像或点云的3D人体姿势估计方法相比，我们利用仅带有2D人体关节注释的弱监督数据和带有3D人体关节注释的完全监督数据。为了减轻监督不力造成的人体姿势歧义，我们采用对抗学习以确保恢复的人体姿势有效。代替在以前的方法中使用深度图像的2D或3D表示，我们利用点云和输入深度图像。我们采用2D CNN从输入的深度图像中提取2D人体关节，2D人体关节帮助我们获得初始3D人体关节并选择有效的采样点，从而可以减少使用点云网络进行3D人体姿势回归的计算成本。使用的点云网络可以缩小网络输入之间的域差距，即点云和3D关节。由于弱监督的对抗学习框架，我们的方法可以从点云中获得准确的3D人体姿势。在ITOP数据集和EVAL数据集上进行的实验表明，我们的方法可以有效地实现最新性能。