So far, most user studies dealing with comprehensibility of machine learning models have used questionnaires or surveys to acquire input from participants. In this article, we argue that compared to questionnaires, the use of an adapted version of a real machine learning interface can yield a new level of insight into what attributes make a machine learning model interpretable, and why. Also, we argue that interpretability research also needs to consider the task of humans editing the model, not least due to the existing or forthcoming legal requirements on the right of human intervention. In this article, we focus on rule models as these are directly interpretable as well as editable. We introduce an extension of the EasyMiner system for generating classification and explorative models based on association rules. The presented web-based rule editing software allows the user to perform common editing actions such as modify rule (add or remove attribute), delete rule, create new rule, or reorder rules. To observe the effect of a particular edit on predictive performance, the user can validate the rule list against a selected dataset using a scoring procedure. The system is equipped with functionality that facilitates its integration with crowdsourcing platforms commonly used to recruit participants.

到目前为止，大多数有关机器学习模型可理解性的用户研究都使用问卷或调查表从参与者那里获取输入。在本文中，我们认为，与问卷调查相比，使用经过修改的真实机器学习界面版本可以使人们对构成机器学习模型的可解释属性以及原因有一个新的认识。同样，我们认为，可解释性研究还需要考虑人类编辑模型的任务，这不仅是由于现有或即将出现的有关人类干预权的法律要求。在本文中，我们将重点放在规则模型上，因为它们可以直接解释和编辑。我们引入EasyMiner系统的扩展，以基于关联规则生成分类和探索性模型。所展示的基于Web的规则编辑软件允许用户执行常见的编辑操作，例如修改规则（添加或删除属性），删除规则，创建新规则或重新排序规则。为了观察特定编辑对预测性能的影响，用户可以使用评分程序针对所选数据集验证规则列表。该系统配备的功能有助于其与通常用于招募参与者的众包平台集成。