Experts in data and physical sciences have to regularly grapple with the problem of competing models. Be it analytical or physics-based models, a cross-cutting challenge for experts is to reliably diagnose which model outcomes appropriately predict or simulate real-world phenomena. Expert judgment involves reconciling information across many, and often, conflicting criteria that describe the quality of model outcomes. In this paper, through a design study with climate scientists, we develop a deeper understanding of the problem and solution space of model diagnostics, resulting in the following contributions: i) a problem and task characterization using which we map experts' model diagnostics goals to multi-way visual comparison tasks, ii) a design space of comparative visual cues for letting experts quickly understand the degree of disagreement among competing models and gauge the degree of stability of model outputs with respect to alternative criteria, and iii) design and evaluation of MyriadCues, an interactive visualization interface for exploring alternative hypotheses and insights about good and bad models by leveraging comparative visual cues. We present case studies and subjective feedback by experts, which validate how MyriadCues enables more transparent model diagnostic mechanisms, as compared to the state of the art.

中文翻译：

---

将小麦与Ch壳分离：比较视觉线索，用于竞争模型的透明诊断。

数据和物理科学领域的专家必须定期应对竞争模型的问题。无论是基于分析的模型还是基于物理的模型，专家的跨领域挑战都是要可靠地诊断出哪种模型结果能够正确预测或模拟现实世界的现象。专家判断涉及调和描述模型结果质量的许多且经常是相互矛盾的标准之间的信息。在本文中，通过与气候科学家进行的设计研究，我们对模型诊断的问题和解决方案空间有了更深入的了解，从而产生了以下贡献：i）问题和任务特征化，我们将专家的模型诊断目标映射到多向视觉比较任务，ii）比较视觉提示的设计空间，使专家可以快速了解竞争模型之间的分歧程度，并根据替代标准评估模型输出的稳定性，并且iii）MyriadCues的设计和评估，这是一种交互式可视化界面通过比较视觉提示来探索关于好模型和坏模型的其他假设和见解。我们提供了案例研究和专家的主观反馈，与现有技术相比，它们验证了MyriadCues如何实现更透明的模型诊断机制。一个交互式的可视化界面，通过利用比较视觉提示来探索关于好的和坏模型的替代假设和见解。我们提供了案例研究和专家的主观反馈，与现有技术相比，它们验证了MyriadCues如何实现更透明的模型诊断机制。一个交互式的可视化界面，通过利用比较视觉提示来探索关于好的和坏模型的替代假设和见解。我们将提供案例研究和专家的主观反馈，与现有技术相比，它们可以验证MyriadCues如何实现更透明的模型诊断机制。