Endowing a companion robot with cognitive abilities to recognize human daily activities, in particular from body skeletons information, is a significant challenge, which needs complex and novel approaches. Recently, most of the proposed approaches exploit the hand-crafted features or the predefined traversal rules techniques to recognize human daily activities from skeleton information, which often lead to the deficit of robustness and generalization. In this work, a novel hybrid framework for human activity-aware robotic system is proposed. In the low-level, a novel Spatio-Temporal Joint based Convolutional Neural Network (STJ-CNN) is proposed to recognize human daily activities in the ambient environments. In the high-level, novel representation and inference services based on Narrative Knowledge Representation Language (NKRL) are proposed to represent and combine the detected human activities with the ambient events, and to infer the semantic context of the detected activity. Empirical experiments on real-world datasets have been conducted, besides an online demonstration created to validate the proposed approach. The final results show that the proposed approach outperforms the baseline models with a significant improvement up to 24% in terms of $F$-score on DAHLIA dataset.

中文翻译：

---

人类活动感知机器人系统的时空卷积网络和 N 元本体

赋予伴侣机器人识别人类日常活动的认知能力，特别是从身体骨骼信息中识别，是一项重大挑战，需要复杂而新颖的方法。最近，大多数提出的方法利用手工制作的特征或预定义的遍历规则技术从骨架信息中识别人类的日常活动，这往往导致鲁棒性和泛化能力的不足。在这项工作中，提出了一种用于人类活动感知机器人系统的新型混合框架。在低层，提出了一种新颖的基于时空联合的卷积神经网络 (STJ-CNN) 来识别周围环境中的人类日常活动。在高层，提出了基于叙事知识表示语言 (NKRL) 的新型表示和推理服务来表示和结合检测到的人类活动与环境事件，并推断检测到的活动的语义上下文。除了为验证所提出的方法而创建的在线演示之外，还对真实世界的数据集进行了实证实验。最终结果表明，所提出的方法优于基线模型，在 DAHLIA 数据集上的 $F$ 得分方面显着提高了 24%。