Developing an intelligent chatbot has evolved in the last few years to become a trending topic in the area of computer science. However, a chatbot often fails to understand the user’s intent, which can lead to the generation of inappropriate responses that cause dialogue breakdown and user dissatisfaction. Detecting the dialogue breakdown is essential to improve the performance of the chatbot and increase user satisfaction. Recent approaches have focused on modeling conversation breakdown using serveral approaches, including supervised and unsupervised approaches. Unsupervised approach relay heavy datasets, which make it challenging to apply it to the breakdown task. Another challenge facing predicting breakdown in conversation is the bias of human annotation for the dataset and the handling process for the breakdown. To tackle this challenge, we have developed a supervised ensemble automated approach that measures Chatbot Quality of Service (CQoS) based on dialogue breakdown. The proposed approach is able to label the datasets based on sentiment considering the context of the conversion to predict the breakdown. In this paper we aim to detect the affect of sentiment change of each speaker in a conversation. Furthermore, we use the supervised ensemble model to measure the CQoS based on breakdown. Then we handle this problem by using a hand-over mechanism that transfers the user to a live agent. Based on this idea, we perform several experiments across several datasets and state-of-the-art models, and we find that using sentiment as a trigger for breakdown outperforms human annotation. Overall, we infer that knowledge acquired from the supervised ensemble model can indeed help to measure CQoS based on detecting the breakdown in conversation.

中文翻译：

---

用于衡量聊天机器人服务质量的监督集成基于情感的框架

开发智能聊天机器人在过去几年中发展成为计算机科学领域的热门话题。然而，聊天机器人往往无法理解用户的意图，这可能会导致产生不适当的响应，从而导致对话中断和用户不满。检测对话中断对于提高聊天机器人的性能和提高用户满意度至关重要。最近的方法侧重于使用多种方法（包括有监督和无监督方法）对对话细分进行建模。无监督方法传递大量数据集，这使得将其应用于故障任务具有挑战性。预测对话中断面临的另一个挑战是数据集人工注释的偏差以及中断的处理过程。为了应对这一挑战，我们开发了一种有监督的集成自动化方法，可以根据对话分解来衡量聊天机器人服务质量 (CQoS)。所提出的方法能够在考虑转换的上下文的情况下基于情绪标记数据集以预测故障。在本文中，我们旨在检测对话中每个说话者的情绪变化的影响。此外，我们使用监督集成模型来衡量基于故障的 CQoS。然后我们通过使用将用户转移到在线代理的切换机制来处理这个问题。基于这个想法，我们在多个数据集和最先进的模型上进行了多次实验，我们发现使用情绪作为故障的触发器优于人工注释。全面的，