Several approaches have been developed to answer specific questions that a user may have about an AI system that can plan and act. However, the problems of identifying which questions to ask and that of computing a user-interpretable symbolic description of the overall capabilities of the system have remained largely unaddressed. This paper presents an approach for addressing these problems by learning user-interpretable symbolic descriptions of the limits and capabilities of a black-box AI system using low-level simulators. It uses a hierarchical active querying paradigm to generate questions and to learn a user-interpretable model of the AI system based on its responses. In contrast to prior work, we consider settings where imprecision of the user's conceptual vocabulary precludes a direct expression of the agent's capabilities. Furthermore, our approach does not require assumptions about the internal design of the target AI system or about the methods that it may use to compute or learn task solutions. Empirical evaluation on several game-based simulator domains shows that this approach can efficiently learn symbolic models of AI systems that use a deterministic black-box policy in fully observable scenarios.

中文翻译：

---

学习用户可解释的黑盒 AI 系统功能描述

已经开发了几种方法来回答用户可能对可以计划和行动的 AI 系统提出的特定问题。然而，确定要问哪些问题以及计算系统整体能力的用户可解释的符号描述的问题在很大程度上仍未得到解决。本文提出了一种通过使用低级模拟器学习用户可解释的黑盒 AI 系统限制和能力的符号描述来解决这些问题的方法。它使用分层主动查询范式来生成问题并根据其响应学习人工智能系统的用户可解释模型。与之前的工作相比，我们考虑了用户概念词汇的不精确性阻碍了代理能力的直接表达的设置。此外，我们的方法不需要对目标 AI 系统的内部设计或它可能用于计算或学习任务解决方案的方法进行假设。对几个基于游戏的模拟器领域的实证评估表明，这种方法可以有效地学习人工智能系统的符号模型，这些模型在完全可观察的场景中使用确定性黑盒策略。