How do we decide the fair value of individual classifiers in an ensemble model? We introduce a new class of transferable utility cooperative games to answer this question. The players in ensemble games are pre-trained binary classifiers that collaborate in an ensemble to correctly label points from a dataset. We design Troupe a scalable algorithm that designates payoffs to individual models based on the Shapley value of those in the ensemble game. We show that the approximate Shapley value of classifiers in these games is an adequate measure for selecting a subgroup of highly predictive models. In addition, we introduce the Shapley entropy a new metric to quantify the heterogeneity of machine learning ensembles when it comes to model quality. We analytically prove that our Shapley value approximation algorithm is accurate and scales to large ensembles and big data. Experimental results on graph classification tasks establish that Troupe gives precise estimates of the Shapley value in ensemble games. We demonstrate that the Shapley value can be used for pruning large ensembles, show that complex classifiers have a prime role in correct and incorrect classification decisions, and provide evidence that adversarial models receive a low valuation.

中文翻译：

---

合奏游戏中分类器的简朴价值

我们如何确定整体模型中各个分类器的公允价值？我们引入了一类新的可转让的实用合作游戏来回答这个问题。合奏游戏中的玩家是经过预训练的二进制分类器，它们在合奏中协作以正确标记数据集中的点。我们设计了Troupe可扩展算法，该算法根据整体游戏中模型的Shapley值为各个模型指定收益。我们显示，在这些游戏中，分类器的近似Shapley值是选择高度预测模型的子组的适当度量。此外，我们为模型质量引入了Shapley熵这一新指标，用于量化机器学习集成的异质性。我们通过分析证明，我们的Shapley值逼近算法是准确的，并且可以缩放到大型集合和大数据。图分类任务的实验结果表明，Troupe可对合奏游戏中Shapley值进行精确估计。我们证明Shapley值可用于修剪大型合奏，表明复杂的分类器在正确和不正确的分类决策中起主要作用，并提供证据表明对抗性模型获得的估值较低。