Classifiers are among the most widely used supervised machine learning algorithms. Many classification models exist, and choosing the right one for a given task is difficult. During model selection and debugging, data scientists need to assess classifiers' performances, evaluate their learning behavior over time, and compare different models. Typically, this analysis is based on single-number performance measures such as accuracy. A more detailed evaluation of classifiers is possible by inspecting class errors. The confusion matrix is an established way for visualizing these class errors, but it was not designed with temporal or comparative analysis in mind. More generally, established performance analysis systems do not allow a combined temporal and comparative analysis of class-level information. To address this issue, we propose ConfusionFlow, an interactive, comparative visualization tool that combines the benefits of class confusion matrices with the visualization of performance characteristics over time. ConfusionFlow is model-agnostic and can be used to compare performances for different model types, model architectures, and/or training and test datasets. We demonstrate the usefulness of ConfusionFlow in a case study on instance selection strategies in active learning. We further assess the scalability of ConfusionFlow and present a use case in the context of neural network pruning.

中文翻译：

---

ConfusionFlow：用于分类器混淆时间分析的模型无关可视化


分类器是最广泛使用的监督机器学习算法之一。存在许多分类模型，为给定任务选择正确的分类模型很困难。在模型选择和调试过程中，数据科学家需要评估分类器的性能，评估其随时间的学习行为，并比较不同的模型。通常，此分析基于单一数字性能测量（例如准确性）。通过检查类别错误可以对分类器进行更详细的评估。混淆矩阵是可视化这些类别错误的既定方法，但其设计时并未考虑到时间或比较分析。更一般地，已建立的绩效分析系统不允许对班级级别信息进行组合的时间和比较分析。为了解决这个问题，我们提出了 ConfusionFlow，这是一种交互式比较可视化工具，它将类混淆矩阵的优点与随时间变化的性能特征可视化结合起来。 ConfusionFlow 与模型无关，可用于比较不同模型类型、模型架构和/或训练和测试数据集的性能。我们在主动学习中实例选择策略的案例研究中展示了 ConfusionFlow 的有用性。我们进一步评估了 ConfusionFlow 的可扩展性，并在神经网络剪枝的背景下提出了一个用例。