The purpose of this study was to develop a method for validation of cognitive models consistent with the remote working situation arising from COVID-19 restrictions in place in Spring 2020. We propose a framework for structuring validation tasks and applying a scoring system to determine initial model validity. We infer an objective validity level for cognitive models requiring no in-person observations, and minimal reliance on remote usability and observational studies. This approach has been derived from the necessity of the COVID-19 response, however, we believe this approach can lower costs and reduce timelines to initial validation in post-Covid-19 studies, enabling faster progress in the development of cognitive engineering systems. A three-stage hybrid validation framework was developed based on existing validation methods and was adapted to enable compliance with the specific limitations derived from COVID-19 response restrictions. This validation method includes elements of argument-based validation combined with a cognitive walkthrough analysis, and reflexivity assessments. We conducted a case study of the proposed framework on a developmental cognitive model of cardiovascular surgery to demonstrate application of a real-world validation task. This framework can be easily and quickly implemented by a small research team and provides a structured validation method to increase confidence in assumptions as well as to provide evidence to support validity claims in the early stages of model development.

中文翻译：

---

开发一种新型混合认知模型验证框架，用于在 COVID-19 限制下实施


本研究的目的是开发一种验证认知模型的方法，该方法与 2020 年春季实施的 COVID-19 限制所产生的远程工作情况相一致。我们提出了一个框架，用于构建验证任务并应用评分系统来确定初始模型有效性。我们推断认知模型的客观有效性水平不需要亲自观察，并且对远程可用性和观察研究的依赖最​​小。这种方法源于应对 COVID-19 的必要性，但是，我们相信这种方法可以降低成本并缩短 Covid-19 后研究中初始验证的时间，从而使认知工程系统的开发取得更快的进展。基于现有验证方法开发了三阶段混合验证框架，并进行了调整，以符合源自 COVID-19 响应限制的特定限制。该验证方法包括基于论证的验证与认知演练分析和反思性评估相结合的要素。我们对所提出的心血管手术发展认知模型框架进行了案例研究，以展示现实世界验证任务的应用。该框架可以由小型研究团队轻松快速地实施，并提供结构化验证方法来增加对假设的信心，并提供证据来支持模型开发早期阶段的有效性主张。