Automatic alignment of brain anatomy in a standard space is a key step when processing magnetic resonance imaging for group analyses. Such brain registration is prone to failure, and the results are therefore typically reviewed visually to ensure quality. There is however no standard, validated protocol available to perform this visual quality control (QC). We propose here a standardized QC protocol for brain registration, with minimal training overhead and no required knowledge of brain anatomy. We validated the reliability of three-level QC ratings (OK, Maybe, Fail) across different raters. Nine experts each rated N = 100 validation images, and reached moderate to good agreement (kappa from 0.4 to 0.68, average of 0.54 ± 0.08), with the highest agreement for “Fail” images (Dice from 0.67 to 0.93, average of 0.8 ± 0.06). We then recruited volunteers through the Zooniverse crowdsourcing platform, and extracted a consensus panel rating for both the Zooniverse raters (N = 41) and the expert raters. The agreement between expert and Zooniverse panels was high (kappa = 0.76). Overall, our protocol achieved a good reliability when performing a two level assessment (Fail vs. OK/Maybe) by an individual rater, or aggregating multiple three-level ratings (OK, Maybe, Fail) from a panel of experts (3 minimum) or non-experts (15 minimum). Our brain registration QC protocol will help standardize QC practices across laboratories, improve the consistency of reporting of QC in publications, and will open the way for QC assessment of large datasets which could be used to train automated QC systems.

中文翻译：

---

功能 MRI 研究中大脑注册高效可靠质量控制的标准化协议


在处理磁共振成像以进行群体分析时，在标准空间中自动对齐大脑解剖结构是关键步骤。这种大脑记录很容易失败，因此通常会通过目视检查结果以确保质量。然而，没有标准的、经过验证的协议可用于执行这种视觉质量控制 (QC)。我们在这里提出了一种用于大脑注册的标准化质量控制协议，以最小的培训开销并且不需要大脑解剖学知识。我们验证了不同评估者的三级 QC 评级（好的、可能、失败）的可靠性。九位专家分别对 N = 100 个验证图像进行评分，并达到中等至良好的一致性（kappa 从 0.4 到 0.68，平均值为 0.54 ± 0.08），“失败”图像的一致性最高（Dice 从 0.67 到 0.93，平均值为 0.8 ± 0.08）。 0.06）。然后，我们通过 Zooniverse 众包平台招募志愿者，并提取 Zooniverse 评分者 (N = 41) 和专家评分者的共识小组评分。专家组和 Zooniverse 小组之间的一致性很高（kappa = 0.76）。总体而言，当由个人评估者执行两级评估（失败与 OK/可能）或汇总专家小组的多个三级评级（OK、可能、失败）（至少 3 个）时，我们的协议实现了良好的可靠性或非专家（至少 15 名）。我们的大脑注册 QC 协议将有助于标准化实验室的 QC 实践，提高出版物中 QC 报告的一致性，并将为可用于训练自动化 QC 系统的大型数据集的 QC 评估开辟道路。