How should reinforcement learning (RL) agents explain themselves to humans not trained in AI? To gain insights into this question, we conducted a 124-participant, four-treatment experiment to compare participants’ mental models of an RL agent in the context of a simple Real-Time Strategy (RTS) game. The four treatments isolated two types of explanations vs. neither vs. both together. The two types of explanations were as follows: (1) saliency maps (an “Input Intelligibility Type” that explains the AI’s focus of attention) and (2) reward-decomposition bars (an “Output Intelligibility Type” that explains the AI’s predictions of future types of rewards). Our results show that a combined explanation that included saliency and reward bars was needed to achieve a statistically significant difference in participants’ mental model scores over the no-explanation treatment. However, this combined explanation was far from a panacea: It exacted disproportionately high cognitive loads from the participants who received the combined explanation. Further, in some situations, participants who saw both explanations predicted the agent’s next action worse than all other treatments’ participants.

中文翻译：

---

普通人的心智模型与强化学习的解释

强化学习 (RL) 代理应如何向未受过 AI 训练的人类解释自己？为了深入了解这个问题，我们进行了一项 124 名参与者、四次处理的实验，以在简单的实时策略 (RTS) 游戏的背景下比较参与者对 RL 代理的心理模型。这四种治疗方法分离出两种类型的解释，两者都没有，两者都结合在一起。两种类型的解释如下：（1）显着性图（一种解释 AI 关注焦点的“输入清晰度类型”）和（2）奖励分解条（一种解释 AI 预测的“输出清晰度类型”）未来的奖励类型）。我们的结果表明，需要包括显着性和奖励条的组合解释才能在参与者的心理模型得分上实现与无解释治疗相比的统计学显着差异。然而，这种综合解释远非灵丹妙药：它对接受综合解释的参与者造成了不成比例的高认知负荷。此外，在某些情况下，看到两种解释的参与者预测代理的下一步行动更差比所有其他治疗的参与者。