We target the problem of sparse 3D reconstruction of dynamic objects observed by multiple unsynchronized video cameras with unknown temporal overlap. To this end, we develop a framework to recover the unknown structure without sequencing information across video sequences. Our proposed compressed sensing framework poses the estimation of 3D structure as the problem of dictionary learning, where the dictionary is defined as an aggregation of the temporally varying 3D structures. Given the smooth motion of dynamic objects, we observe any element in the dictionary can be well approximated by a sparse linear combination of other elements in the same dictionary (i.e., self-expression). Our formulation optimizes a biconvex cost function that leverages a compressed sensing formulation and enforces both structural dependency coherence across video streams, as well as motion smoothness across estimates from common video sources. We further analyze the reconstructability of our approach under different capture scenarios, and its comparison and relation to existing methods. Experimental results on large amounts of synthetic data as well as real imagery demonstrate the effectiveness of our approach.

中文翻译：

---

用于动态 3D 重建的自我表达词典学习


我们的目标是通过具有未知时间重叠的多个不同步摄像机观察到的动态对象的稀疏 3D 重建问题。为此，我们开发了一个框架来恢复未知结构，而无需跨视频序列对信息进行排序。我们提出的压缩感知框架将 3D 结构的估计视为字典学习的问题，其中字典被定义为随时间变化的 3D 结构的聚合。考虑到动态对象的平滑运动，我们观察到字典中的任何元素都可以通过同一字典中其他元素的稀疏线性组合来很好地近似（即自我表达）。我们的公式优化了双凸成本函数，该函数利用压缩感知公式并强制跨视频流的结构依赖一致性，以及来自常见视频源的估计的运动平滑度。我们进一步分析了我们的方法在不同捕获场景下的可重构性，及其与现有方法的比较和关系。大量合成数据和真实图像的实验结果证明了我们方法的有效性。