Purpose

Where multiple in silico tools are concordant, the American College of Medical Genetics and Genomics/Association for Molecular Pathology (ACMG/AMP) framework affords supporting evidence toward pathogenicity or benignity, equivalent to a likelihood ratio of ~2. However, limited availability of “clinical truth sets” and prior use in tool training limits their utility for evaluation of tool performance.

Methods

We created a truth set of 9,436 missense variants classified as deleterious or tolerated in clinically validated high-throughput functional assays for BRCA1, BRCA2, MSH2, PTEN, and TP53 to evaluate predictive performance for 44 recommended/commonly used in silico tools.

Results

Over two-thirds of the tool–threshold combinations examined had specificity of <50%, thus substantially overcalling deleteriousness. REVEL scores of 0.8–1.0 had a Positive Likelihood Ratio (PLR) of 6.74 (5.24–8.82) compared to scores <0.7 and scores of 0–0.4 had a Negative Likelihood Ratio (NLR) of 34.3 (31.5–37.3) compared to scores of >0.7. For Meta-SNP, the equivalent PLR = 42.9 (14.4–406) and NLR = 19.4 (15.6–24.9).

Conclusion

Against these clinically validated “functional truth sets," there was wide variation in the predictive performance of commonly used in silico tools. Overall, REVEL and Meta-SNP had best balanced accuracy and might potentially be used at stronger evidence weighting than current ACMG/AMP prescription, in particular for predictions of benignity.

中文翻译：

---

44 种计算机工具针对癌症易感基因的多个大规模功能测定的临床似然比和平衡准确度

目的

在多个计算机工具一致的情况下，美国医学遗传学和基因组学/分子病理学协会 (ACMG/AMP) 框架为致病性或良性提供了支持证据，相当于似然比约为 2。然而，“临床真相集”的有限可用性和先前在工具训练中的使用限制了它们在评估工具性能方面的效用。

方法

我们创建了一个包含 9,436 个错义变体的真值集，这些错义变体在经临床验证的BRCA1、BRCA2、MSH2、PTEN和TP53高通量功能分析中被归类为有害或耐受，以评估 44 种推荐/常用的计算机工具的预测性能。

结果

超过三分之二的工具-阈值组合具有<50%的特异性，因此大大超过了有害性。REVEL 得分为 0.8-1.0 的正似然比 (PLR) 为 6.74 (5.24-8.82)，而得分 <0.7，0-0.4 的负似然比 (NLR) 为 34.3 (31.5-37.3) 与得分相比> 0.7。对于 Meta-SNP，等效 PLR = 42.9 (14.4–406) 和 NLR = 19.4 (15.6–24.9)。

结论

针对这些经过临床验证的“功能性真相集”，常用计算机工具的预测性能存在很大差异。总体而言，REVEL 和 Meta-SNP 具有最佳平衡精度，并且可能用于比当前 ACMG/AMP 更强的证据权重处方，特别是对于良性的预测。