In the era of "information overload", effective information provision is essential for enabling rapid response and critical decision making. In making sense of diverse information sources, data dashboards have become an indispensable tool, providing fast, effective, adaptable, and personalized access to information for professionals and the general public alike. However, these objectives place a heavy requirement on dashboards as information systems, resulting in poor usability and ineffective design. Understanding these shortfalls is a challenge given the absence of a consistent and comprehensive approach to dashboard evaluation. In this paper we systematically review literature on dashboard implementation in the healthcare domain, a field where dashboards have been employed widely, and in which there is widespread interest for improving the current state of the art, and subsequently analyse approaches taken towards evaluation. We draw upon consolidated dashboard literature and our own observations to introduce a general definition of dashboards which is more relevant to current trends, together with a dashboard task-based classification, which underpin our subsequent analysis. From a total of 81 papers we derive seven evaluation scenarios - task performance, behaviour change, interaction workflow, perceived engagement, potential utility, algorithm performance and system implementation. These scenarios distinguish different evaluation purposes which we illustrate through measurements, example studies, and common challenges in evaluation study design. We provide a breakdown of each evaluation scenario, and highlight some of the subtle and less well posed questions. We conclude by outlining a number of active discussion points and a set of dashboard evaluation best practices for the academic, clinical and software development communities alike.

中文翻译：

---

用于评估医疗保健仪表板的框架

在“信息过载”时代，有效的信息提供对于实现快速响应和关键决策至关重要。在理解各种信息源方面，数据仪表板已成为不可或缺的工具，为专业人士和普通大众提供快速、有效、适应性强和个性化的信息访问。然而，这些目标对仪表板作为信息系统提出了很高的要求，导致可用性差和设计效率低下。鉴于缺乏一致且全面的仪表板评估方法，了解这些不足是一项挑战。在本文中，我们系统地回顾了医疗保健领域仪表板实施的文献，该领域仪表板已被广泛采用，并且对改进当前技术状态存在广泛兴趣，并随后分析评估方法。我们利用综合仪表板文献和我们自己的观察来介绍与当前趋势更相关的仪表板的一般定义，以及基于仪表板任务的分类，这是我们后续分析的基础。从总共 81 篇论文中，我们得出了七个评估场景——任务性能、行为改变、交互工作流、感知参与度、潜在效用、算法性能和系统实现。这些情景区分了不同的评估目的，我们通过测量、示例研究和评估研究设计中的常见挑战来说明这些目的。我们提供了每个评估场景的细分，并突出一些微妙的和不太好的问题。最后，我们概述了一些活跃的讨论点和一组面向学术、临床和软件开发社区的仪表板评估最佳实践。