Intelligent systems and advanced automation are involved in information collection and evaluation, decision-making, and the implementation of chosen actions. In such systems, human responsibility becomes equivocal. Understanding human causal responsibility is particularly important when systems can harm people, as with autonomous vehicles or, most notably, with autonomous weapon systems (AWSs). Using information theory, we developed a responsibility quantification (ResQu) model of human causal responsibility in intelligent systems and demonstrated its applications on decisions regarding AWS. The analysis reveals that comparative human responsibility for outcomes is often low, even when major functions are allocated to the human. Thus, broadly stated policies of keeping humans in the loop and having meaningful human control are misleading and cannot truly direct decisions on how to involve humans in advanced automation. The current model assumes stationarity, full knowledge regarding the characteristic of the human and automation, and ignores temporal aspects. It is an initial step toward the development of a comprehensive responsibility model that will make it possible to quantify human causal responsibility. The model can serve as an additional tool in the analysis of system design alternatives and policy decisions regarding human causal responsibility, providing a novel, quantitative perspective on these matters. Note to Practitioners —We developed a theoretical model and a quantitative measure for computing the comparative human causal responsibility in the interaction with intelligent systems and advanced automation. Our responsibility measure can be applied by practitioners (system designers, regulators, and so on) to estimate user responsibility in specific system configurations. This can serve as an additional tool in the comparison between alternative system designs or deployment policies, by relating different automation design options to their predicted effect on the users’ responsibility. To apply the model (which is based on entropy and mutual information) to real-world systems, one must deduce the underlying distributions, either from known system properties or from empirical observations, taken over time. The initial version of the model we present here assumes that the combined human–automation system is stationary and ergodic. Real-world systems may not be stationary and ergodic or cannot be observed sufficiently to allow accurate estimates of the required input of multivariate probabilities, in which case the computed responsibility values should be treated with caution. Nevertheless, the construction of a ResQu information flow model, combined with sensitivity analyses of how changes in the input probabilities and assumptions affect the responsibility measure, will often reveal important qualitative properties and supply valuable insights regarding the general level of meaningful human involvement and comparative responsibility in a system.

中文翻译：

---

人与自动化互动的责任量化模型

智能系统和高级自动化涉及信息收集和评估，决策以及选定动作的实施。在这样的系统中，人类责任变得模棱两可。当系统可能会伤害人时，例如自动驾驶汽车，或者最明显的是使用自动武器系统（AWS），了解人为原因责任尤为重要。使用信息论，我们开发了智能系统中人类因果责任的责任量化（ResQu）模型，并演示了其在有关AWS决策中的应用。分析表明，即使将主要职能分配给了人，比较的人对结果的责任通常较低。从而，广为人知的使人处于循环中并进行有意义的人为控制的政策具有误导性，无法真正指导如何使人参与高级自动化的决策。当前模型假定平稳性，关于人员和自动化特性的全面知识，并且忽略了时间方面。这是迈向全面责任模型发展的第一步，它将使量化人为因果责任成为可能。该模型可以用作分析系统设计备选方案和有关人为责任的政策决策的附加工具，从而提供有关这些问题的新颖，定量的观点。关于人的特征和自动化的全部知识，并且忽略时间方面。这是迈向全面责任模型发展的第一步，它将使量化人为因果责任成为可能。该模型可以用作分析系统设计备选方案和有关人为责任的政策决策的附加工具，从而提供有关这些问题的新颖，定量的观点。关于人的特征和自动化的全部知识，并且忽略时间方面。这是迈向全面责任模型发展的第一步，它将使量化人为因果责任成为可能。该模型可以用作分析系统设计备选方案和有关人为责任的政策决策的附加工具，从而提供有关这些问题的新颖，定量的观点。执业者注意 —我们开发了一种理论模型和定量方法，用于计算与智能系统和先进自动化的交互作用中相对的人类因果责任。从业人员（系统设计人员，监管人员等）可以采用我们的责任度量，以估计特定系统配置中的用户责任。通过将不同的自动化设计选项与其对用户职责的预期影响相关联，这可以用作替代系统设计或部署策略之间比较的附加工具。要将模型（基于熵和互信息）应用于现实世界系统，必须从已知的系统属性或经验观察中推断出随时间推移的潜在分布。我们在此处介绍的模型的初始版本假定组合的人工自动化系统是固定的和遍历的。现实世界的系统可能不是固定的，遍历遍历的，或者不能被充分观察以允许准确估计所需的多元概率输入，在这种情况下，应谨慎对待计算的责任值。尽管如此，ResQu信息流模型的构建，结合对输入概率和假设的变化如何影响责任度量的敏感性分析，通常会揭示重要的定性属性，并提供有关有意义的人类参与和比较责任的总体水平的有价值的见解。在一个系统中。现实世界中的系统可能不是固定的，遍历遍历的，或者不能被充分观察以允许准确估计所需的多元概率输入，在这种情况下，应谨慎对待计算出的责任值。尽管如此，ResQu信息流模型的构建，结合对输入概率和假设的变化如何影响责任度量的敏感性分析，通常会揭示重要的定性属性，并提供有关有意义的人类参与和比较责任的总体水平的有价值的见解。在一个系统中。现实世界中的系统可能不是固定的，遍历遍历的，或者不能被充分观察以允许准确估计所需的多元概率输入，在这种情况下，应谨慎对待计算出的责任值。尽管如此，ResQu信息流模型的构建，结合对输入概率和假设的变化如何影响责任度量的敏感性分析，通常会揭示重要的定性属性，并提供有关有意义的人类参与和比较责任的总体水平的有价值的见解。在一个系统中。