When developing a product, it is important to consider product performance from a user perspective. This type of evaluation can be done through operational testing—assessing the ability of representative users to satisfactorily accomplish tasks with the product in operationally representative environments. This process can be expensive and time-consuming, but is critical to understanding whether the product can adequately do the job for which it was designed. We show how an existing design of experiments (DOEs) process for operational testing can be leveraged to construct a Bayesian adaptive design. This design, nested within the larger design created by the DOE process, allows interim analyses to stop testing early for success or futility. Representative simulations are presented to demonstrate how these interim analyses can be used in an operational test setting, and reductions in necessary test events are shown. The application of Bayesian-adaptive design methods will allow future operational testing to be conducted in less time and at less expense, on average, without compromising the ability of the existing process to verify the product meets the user's needs.

中文翻译：

---

通过贝叶斯自适应设计提高操作测试效率的框架

在开发产品时，从用户的角度考虑产品性能很重要。这种类型的评估可以通过操作测试来完成——评估代表性用户在具有代表性的操作环境中令人满意地完成产品任务的能力。这个过程可能既昂贵又耗时，但对于了解产品是否能够充分完成其设计工作至关重要。我们展示了如何利用现有的用于操作测试的实验设计 (DOE) 过程来构建贝叶斯自适应设计。这种设计嵌套在由 DOE 过程创建的更大设计中，允许临时分析提前停止测试是否成功。展示了具有代表性的模拟来演示如何在操作测试设置中使用这些临时分析，并显示了必要测试事件的减少。贝叶斯自适应设计方法的应用将使未来的操作测试能够以更短的时间和更少的费用进行，平均而言，不会影响现有流程验证产品满足用户需求的能力。