We present LEGOEval, an open-source toolkit that enables researchers to easily evaluate dialogue systems in a few lines of code using the online crowdsource platform, Amazon Mechanical Turk. Compared to existing toolkits, LEGOEval features a flexible task design by providing a Python API that maps to commonly used React.js interface components. Researchers can personalize their evaluation procedures easily with our built-in pages as if playing with LEGO blocks. Thus, LEGOEval provides a fast, consistent method for reproducing human evaluation results. Besides the flexible task design, LEGOEval also offers an easy API to review collected data.

中文翻译：

---

LEGOEval：通过众包进行对话系统评估的开源工具包

我们介绍了LEGOEval，这是一个开放源代码工具包，使研究人员可以使用在线众筹平台Amazon Mechanical Turk用几行代码轻松评估对话系统。与现有工具包相比，LEGOEval通过提供可映射到常用React.js接口组件的Python API，具有灵活的任务设计。研究人员可以使用我们的内置页面轻松地个性化他们的评估程序，就像玩乐高积木一样。因此，LEGOEval提供了一种快速，一致的方法来重现人类评估结果。除了灵活的任务设计外，LEGOEval还提供了一个简单的API来查看收集的数据。