A growing number of approaches exist to generate explanations for image classification. However, few of these approaches are subjected to human-subject evaluations, partly because it is challenging to design controlled experiments with natural image datasets, as they leave essential factors out of the researcher's control. With our approach, researchers can describe their desired dataset with only a few parameters. Based on these, our library generates synthetic image data of two 3D abstract animals. The resulting data is suitable for algorithmic as well as human-subject evaluations. Our user study results demonstrate that our method can create biases predictive enough for a classifier and subtle enough to be noticeable only to every second participant inspecting the data visually. Our approach significantly lowers the barrier for conducting human subject evaluations, thereby facilitating more rigorous investigations into interpretable machine learning. For our library and datasets see, https://github.com/mschuessler/two4two/

中文翻译：

---

Two4Two：评估可解释的机器学习-用于受控实验的综合数据集

存在越来越多的方法来生成图像分类的解释。但是，这些方法中很少有人进行人体评估，部分原因是使用自然图像数据集设计受控实验具有挑战性，因为它们将基本因素置于研究人员的控制范围之外。使用我们的方法，研究人员仅需几个参数即可描述所需的数据集。基于这些，我们的库生成了两个3D抽象动物的合成图像数据。所得数据适用于算法评估和人类受试者评估。我们的用户研究结果表明，我们的方法可以为分类器创建足够可预测的偏差，并且仅在视觉上检查数据的每一秒参与者中就可以发现足够细微的偏差。我们的方法大大降低了进行人类主题评估的障碍，从而有助于对可解释的机器学习进行更严格的研究。有关我们的库和数据集，请参见https://github.com/mschuessler/two4two/