Reminiscence is a lifelong activity that happens throughout our lifespan. While memories can serve as topics in people’s everyday conversations, recalling the past can also help us build self-esteem and increase our level of happiness. In this paper, we aim to develop a robot companion that helps people to recollect their memories from personal photographs. We focus on how a robot can associate concepts relevant to the content in the photographs and evoke people’s memories by asking questions that are both relatable and engaging. To understand the content of a picture, we applied deep learning techniques in order to recognize events, objects, and scenes in it. Then, these observations and any user utterances are considered in a Markov random field (MRF)-based algorithm that contains common sense knowledge of a number of events, with loopy belief propagation being used to infer possible associated concepts and topics. Afterward, the robot poses appropriate questions about the selected topics, guiding the user to reminisce through conversation. Our results show that the proposed system can pose related and appropriate questions to interact with the user, and has the potential guide the user to recall the past in an organized way.

中文翻译：

---

机器人辅助个人照片回忆的交互式提问系统

回忆是一种终生的活动，在我们的一生中都会发生。虽然回忆可以作为人们日常对话的话题，但回忆过去也可以帮助我们建立自尊并提高我们的幸福水平。在本文中，我们的目标是开发一种机器人伴侣，帮助人们从个人照片中回忆起他们的记忆。我们专注于机器人如何将与照片内容相关的概念关联起来，并通过提出既相关又引人入胜的问题来唤起人们的记忆。为了理解图片的内容，我们应用了深度学习技术来识别其中的事件、对象和场景。然后，在基于马尔可夫随机场 (MRF) 的算法中考虑这些观察结果和任何用户话语，该算法包含许多事件的常识知识，使用循环信念传播来推断可能的相关概念和主题。之后，机器人会针对所选主题提出适当的问题，引导用户通过对话进行回忆。我们的结果表明，所提出的系统可以提出相关和适当的问题来与用户互动，并有可能引导用户以有组织的方式回忆过去。