Abstract Quality of life (QoL) is an effective index of well-being, including physical health, aspect of social activity, and mental state of individuals. A new approach that uses a deep-learning architecture to estimate the score of a user's QoL is presented. This system was built using a combination of a 3D convolutional neural network and a support vector machine for multimodal data. In order to evaluate the accuracy of the estimation system, three experiments were conducted. Before these experiments, ten hours of audio and video data were collected from healthy participants during a natural-language conversation with a conversational agent we implemented. In the first experiment, the QoL question-answer estimation experiment, the accuracy of “Physical functioning,” which is one of the eight scales that constitute QoL, reached 84.0%. In the second experiment, the QoL-score-regression experiment, in which the scores of each scale were directly estimated, the distribution of the difference between the actual score and the estimated results, known as error, was investigated. These results imply that the features necessary for QoL estimation can be extracted from audio and video data, except for the “Mental Health” domain. One of the reasons why it was difficult to estimate the “Mental Health” scale may be that the learning framework could not extract an appropriate feature for estimation. Therefore, we estimated “Mental Health” by focusing on eye movement. From the result, it was proven that estimation is possible, and the proposed system using multimodal data demonstrated its effectiveness for estimation for all eight scales that constitute QoL and for extracting high-dimensional information regarding the QoL of a human, including their satisfaction level towards daily life and social activities. Finally, suggestions and discussions regarding the plausible behavior of the estimation results were made from the viewpoint of human–agent interaction in the field of elderly welfare.

中文翻译：

---

基于 3D 卷积神经网络的用于估计生活质量的新型远程护理方法

摘要 生活质量（QoL）是一种有效的幸福指数，包括个体的身体健康、社会活动方面和精神状态。提出了一种使用深度学习架构来估计用户 QoL 分数的新方法。该系统是使用 3D 卷积神经网络和多模态数据支持向量机的组合构建的。为了评估估计系统的准确性，进行了三个实验。在这些实验之前，在与我们实施的对话代理进行自然语言对话期间，从健康参与者那里收集了 10 小时的音频和视频数据。在第一个实验中，QoL 问答估计实验中，构成 QoL 的八个量表之一的“身体机能”的准确率达到了 84.0%。在第二个实验中，QoL-score-regression 实验中，直接估计每个量表的分数，调查实际分数与估计结果之间的差异分布，称为误差。这些结果意味着除了“心理健康”领域之外，可以从音频和视频数据中提取 QoL 估计所需的特征。“心理健康”量表难以估计的原因之一可能是学习框架无法提取合适的特征进行估计。因此，我们通过关注眼球运动来估计“心理健康”。结果证明，估计是可能的，所提出的使用多模态数据的系统证明了其对构成 QoL 的所有八个尺度的估计以及提取有关人类 QoL 的高维信息的有效性，包括他们对日常生活和社交活动的满意度。最后，从老年人福利领域人机交互的角度对估计结果的合理行为提出了建议和讨论。