To reduce program risks, engineering methods capitalizing on modeling and machine assistance have been extensively investigated within systems engineering (and more specifically requirements engineering) literature over the past 20 years. However, there are few quantitative comparisons between model-based approaches and legacy document-centric approaches. Studies have shown that the lack of data regarding improvements of modeling has decelerated the adoption of model-based practices. To help address this gap, the authors conducted a screening experiment to compare the effects of modeling on an engineer’s ability to determine if a system has met its originating requirements. First, a notional acquisition program was created based on an unmanned aerial system, including originating requirements as well as both document and model-based design artifacts. Requirements were captured in both a traditional requirements document and a goal oriented requirements engineering model. System implementation data were capturing in both traditional document artifacts and a SysML model. Participants of varying experience levels used randomized combinations of document and model-based approaches to determine whether the notional system met its requirements. The experiment measured the review duration and accuracy, permitting analysis of the effects of model-based approaches for both requirements and system implementation data. The results of the experiment showed that use of a requirements model did not statistically effect the review. A system implementation model was shown to improve novice participants’ reviews, but did not statistically effect experienced participants. The results of this study should inform future research on the use of models, particularly the return on the modeling investment.

为了降低程序风险，在过去 20 年中，系统工程（更具体地说是需求工程）文献中广泛研究了利用建模和机器辅助的工程方法。然而，基于模型的方法和传统的以文档为中心的方法之间的定量比较很少。研究表明，缺乏有关建模改进的数据已经减缓了基于模型的实践的采用。为了帮助解决这一差距，作者进行了一项筛选实验，以比较建模对工程师确定系统是否满足其原始要求的能力的影响。首先，基于无人机系统创建了一个名义上的采购计划，包括原始需求以及基于文档和模型的设计工件。在传统的需求文档和面向目标的需求工程模型中都捕获了需求。系统实现数据在传统文档工件和 SysML 模型中都被捕获。不同经验水平的参与者使用基于文档和模型的方法的随机组合来确定概念系统是否满足其要求。该实验测量了审查持续时间和准确性，允许分析基于模型的方法对需求和系统实施数据的影响。实验结果表明，需求模型的使用在统计上不会影响评审。一个系统实施模型被证明可以提高新手参与者的评论，但对有经验的参与者没有统计学影响。这项研究的结果应该为未来关于模型使用的研究提供信息，特别是建模投资的回报。