This document presents a detailed description of the challenge on clarifying questions for dialogue systems (ClariQ). The challenge is organized as part of the Conversational AI challenge series (ConvAI3) at Search Oriented Conversational AI (SCAI) EMNLP workshop in 2020. The main aim of the conversational systems is to return an appropriate answer in response to the user requests. However, some user requests might be ambiguous. In IR settings such a situation is handled mainly thought the diversification of the search result page. It is however much more challenging in dialogue settings with limited bandwidth. Therefore, in this challenge, we provide a common evaluation framework to evaluate mixed-initiative conversations. Participants are asked to rank clarifying questions in an information-seeking conversations. The challenge is organized in two stages where in Stage 1 we evaluate the submissions in an offline setting and single-turn conversations. Top participants of Stage 1 get the chance to have their model tested by human annotators.

中文翻译：

---

ConvAI3：为开放域对话系统生成澄清问题 (ClariQ)

本文档详细描述了澄清对话系统 (ClariQ) 问题的挑战。该挑战是 2020 年面向搜索的对话人工智能 (SCAI) EMNLP 研讨会的对话人工智能挑战系列 (ConvAI3) 的一部分。对话系统的主要目标是响应用户请求返回适当的答案。但是，某些用户请求可能不明确。在 IR 设置中，这种情况的处理主要是考虑了搜索结果页面的多样化。然而，在带宽有限的对话设置中更具挑战性。因此，在这个挑战中，我们提供了一个通用的评估框架来评估混合主动对话。要求参与者在寻求信息的对话中对澄清问题进行排名。挑战分为两个阶段，在第一阶段，我们评估离线设置和单轮对话中的提交。第一阶段的顶级参与者有机会让人工注释者测试他们的模型。