Advances in AI technologies have resulted in superior levels of AI-based model performance. However, this has also led to a greater degree of model complexity, resulting in 'black box' models. In response to the AI black box problem, the field of explainable AI (xAI) has emerged with the aim of providing explanations catered to human understanding, trust, and transparency. Yet, we still have a limited understanding of how xAI addresses the need for explainable AI in the context of healthcare. Our research explores the differing explanation needs amongst stakeholders during the development of an AI-system for classifying COVID-19 patients for the ICU. We demonstrate that there is a constellation of stakeholders who have different explanation needs, not just the 'user'. Further, the findings demonstrate how the need for xAI emerges through concerns associated with specific stakeholder groups i.e., the development team, subject matter experts, decision makers, and the audience. Our findings contribute to the expansion of xAI by highlighting that different stakeholders have different explanation needs. From a practical perspective, the study provides insights on how AI systems can be adjusted to support different stakeholders needs, ensuring better implementation and operation in a healthcare context.

中文翻译：

---

可解释的 AI，但可向谁解释？

人工智能技术的进步导致了基于人工智能的模型性能的卓越水平。然而，这也导致了更大程度的模型复杂性，从而产生了“黑盒”模型。为了应对人工智能黑盒问题，可解释人工智能 (xAI) 领域应运而生，旨在提供满足人类理解、信任和透明度的解释。然而，我们对 xAI 如何满足医疗保健背景下对可解释 AI 的需求仍知之甚少。我们的研究探索了在开发用于对 ICU 的 COVID-19 患者进行分类的 AI 系统期间利益相关者之间的不同解释需求。我们证明有一群具有不同解释需求的利益相关者，而不仅仅是“用户”。更多，调查结果展示了对 xAI 的需求是如何通过与特定利益相关者群体（即开发团队、主题专家、决策者和观众）相关的关注而出现的。我们的发现通过强调不同的利益相关者有不同的解释需求来促进 xAI 的扩展。从实践的角度来看，该研究提供了有关如何调整 AI 系统以支持不同利益相关者的需求，确保在医疗保健环境中更好地实施和运营的见解。