In this paper, we survey the methods and concepts developed for the evaluation of dialogue systems. Evaluation, in and of itself, is a crucial part during the development process. Often, dialogue systems are evaluated by means of human evaluations and questionnaires. However, this tends to be very cost- and time-intensive. Thus, much work has been put into finding methods which allow a reduction in involvement of human labour. In this survey, we present the main concepts and methods. For this, we differentiate between the various classes of dialogue systems (task-oriented, conversational, and question-answering dialogue systems). We cover each class by introducing the main technologies developed for the dialogue systems and then present the evaluation methods regarding that class.

中文翻译：

---

对话系统评价方法调查

在本文中，我们调查了为评估对话系统而开发的方法和概念。评估本身是开发过程中的关键部分。通常，对话系统是通过人工评估和问卷来评估的。然而，这往往非常耗费成本和时间。因此，已经投入了大量工作来寻找允许减少人类劳动参与的方法。在本次调查中，我们介绍了主要概念和方法。为此，我们区分了各类对话系统（面向任务、对话和问答对话系统）。我们通过介绍为对话系统开发的主要技术来涵盖每个类，然后介绍有关该类的评估方法。