Context

Demonstrating high reliability and safety for safety-critical systems (SCSs) remains a hard problem. Diverse evidence needs to be combined in a rigorous way: in particular, results of operational testing with other evidence from design and verification. Growing use of machine learning in SCSs, by precluding most established methods for gaining assurance, makes evidence from operational testing even more important for supporting safety and reliability claims.

Objective

We revisit the problem of using operational testing to demonstrate high reliability. We use Autonomous Vehicles (AVs) as a current example. AVs are making their debut on public roads: methods for assessing whether an AV is safe enough are urgently needed. We demonstrate how to answer 5 questions that would arise in assessing an AV type, starting with those proposed by a highly-cited study.

Method

We apply new theorems extending our Conservative Bayesian Inference (CBI) approach, which exploit the rigour of Bayesian methods while reducing the risk of involuntary misuse associated (we argue) with now-common applications of Bayesian inference; we define additional conditions needed for applying these methods to AVs.

Results

Prior knowledge can bring substantial advantages if the AV design allows strong expectations of safety before road testing. We also show how naive attempts at conservative assessment may lead to over-optimism instead; why extrapolating the trend of disengagements (take-overs by human drivers) is not suitable for safety claims; use of knowledge that an AV has moved to a “less stressful” environment.

Conclusion

While some reliability targets will remain too high to be practically verifiable, our CBI approach removes a major source of doubt: it allows use of prior knowledge without inducing dangerously optimistic biases. For certain ranges of required reliability and prior beliefs, CBI thus supports feasible, sound arguments. Useful conservative claims can be derived from limited prior knowledge.

中文翻译：

---

通过运行测试评估关键安全系统：自动驾驶汽车研究

语境

展示安全关键系统（SCS）的高可靠性和安全性仍然是一个难题。需要将各种证据严格地结合起来：特别是，将操作测试的结果与设计和验证中的其他证据相结合。通过排除大多数已建立的获得保证的方法，SCS中越来越多地使用机器学习，这使得操作测试的证据对于支持安全性和可靠性声明更为重要。

目的

我们重新讨论使用操作测试来证明高可靠性的问题。我们以自动驾驶汽车（AVs）为例。AV在公路上首次亮相：迫切需要一种评估AV是否足够安全的方法。我们将演示如何回答由高被引研究提出的5个问题，这些问题将在评估AV类型时出现。

方法

我们应用新的定理扩展了保守的贝叶斯推理（CBI）方法，该方法利用贝叶斯方法的严格性，同时降低了与贝叶斯推理的常见应用（我们认为）相关的非自愿滥用的风险；我们定义了将这些方法应用于AV所需的其他条件。

结果

如果视音频设计允许在路试之前强烈期望安全，那么先验知识可以带来实质性的优势。我们还展示了对保守评估的天真的尝试可能会导致过度乐观的情况。为什么推断脱离趋势（驾驶员接管）不适合安全性要求；使用有关AV已转移到“压力较小”环境的知识。

结论

尽管某些可靠性目标仍然太高而无法实际验证，但我们的CBI方法消除了一个主要的疑问来源：它允许使用先验知识而不会引起危险的乐观偏见。对于某些范围的所需可靠性和先前的信念，CBI因此支持可行的合理论证。有用的保守主张可以从有限的先验知识中得出。