There is a large variation in the activities that humans perform in their everyday lives. We consider modeling these composite human activities which comprises multiple basic level actions in a completely unsupervised setting. Our model learns high-level co-occurrence and temporal relations between the actions. We consider the video as a sequence of short-term action clips, which contains human-words and object-words. An activity is about a set of action-topics and object-topics indicating which actions are present and which objects are interacting with. We then propose a new probabilistic model relating the words and the topics. It allows us to model long-range action relations that commonly exist in the composite activities, which is challenging in previous works. We apply our model to the unsupervised action segmentation and clustering, and to a novel application that detects forgotten actions, which we call action patching. For evaluation, we contribute a new challenging RGB-D activity video dataset recorded by the new Kinect v2, which contains several human daily activities as compositions of multiple actions interacting with different objects. Moreover, we develop a robotic system that watches and reminds people using our action patching algorithm. Our robotic setup can be easily deployed on any assistive robots.

中文翻译：

---

Watch-n-Patch：无监督的行动和关系学习

人类在日常生活中进行的活动存在很大差异。我们考虑对这些复合人类活动进行建模，这些活动包括在完全无人监督的情况下进行的多个基本级别的动作。我们的模型学习动作之间的高层共现和时间关系。我们将视频视为一系列短期动作剪辑，其中包含人为词和宾语。活动是关于一组动作主题和对象主题的，指示存在哪些动作以及与哪些对象进行交互。然后，我们提出一个与单词和主题相关的新概率模型。它使我们能够对复合活动中通常存在的远程动作关系进行建模，这在以前的工作中具有挑战性。我们将模型应用于无监督的行动细分和聚类，以及一种新颖的应用程序，它可以检测到被遗忘的动作，我们将其称为动作补丁。为了进行评估，我们贡献了一个新的具有挑战性的RGB-D活动视频数据集，该数据集是由新的Kinect v2录制的，其中包含一些人类日常活动，这些活动是与不同对象相互作用的多个动作的组合。此外，我们开发了一种机器人系统，该机器人系统使用我们的动作修补算法来监视并提醒人们。我们的机器人设置可以轻松地部署在任何辅助机器人上。我们开发了一种机器人系统，可以使用我们的动作修补算法来监视并提醒人们。我们的机器人设置可以轻松地部署在任何辅助机器人上。我们开发了一种机器人系统，可以使用我们的动作修补算法来监视并提醒人们。我们的机器人设置可以轻松地部署在任何辅助机器人上。