We discuss challenges and strategies for evaluating natural language interfaces (NLIs) for data visualization. Through an examination of prior studies and reflecting on own experiences in evaluating visualization NLIs, we highlight benefits and considerations of three task framing strategies: Jeopardy-style facts, open-ended tasks, and target replication tasks. We hope the discussions in this article can guide future researchers working on visualization NLIs and help them avoid common challenges and pitfalls when evaluating these systems. Finally, to motivate future research, we highlight topics that call for further investigation including development of new evaluation metrics, and considering the type of natural language input (spoken versus typed), among others.

中文翻译：

---

如何问该说什么？：评估数据可视化自然语言界面的策略


我们讨论评估数据可视化自然语言接口 (NLI) 的挑战和策略。通过检查先前的研究并反思自己在评估可视化 NLI 方面的经验，我们强调了三种任务框架策略的好处和考虑因素：危险型事实、开放式任务和目标复制任务。我们希望本文中的讨论能够指导未来从事可视化 NLI 工作的研究人员，并帮助他们在评估这些系统时避免常见的挑战和陷阱。最后，为了激励未来的研究，我们强调需要进一步研究的主题，包括开发新的评估指标，以及考虑自然语言输入的类型（口语与打字）等。