Background During the past two decades, the number and complexity of clinical trials have risen dramatically increasing the difficulty of choosing sites for inspection. FDA’s resources are limited and so sites should be chosen with care.

Purpose To determine if data mining techniques and/or unsupervised statistical monitoring can assist with the process of identifying potential clinical sites for inspection.

Methods Five summary-level clinical site datasets from four new drug applications (NDA) and one biologics license application (BLA), where the FDA had performed or had planned site inspections, were used. The number of sites inspected and the results of the inspections were blinded to the researchers. Five supervised learning models from the previous two years (2016–2017) of an on-going research project were used to predict site inspections results, i.e., No Action Indicated (NAI), Voluntary Action Indicated (VAI), or Official Action Indicated (OAI). Statistical Monitoring Applied to Research Trials (SMART^TM) software for unsupervised statistical monitoring software developed by CluePoints (Mont-Saint-Guibert, Belgium) was utilized to identify atypical centers (via a p-value approach) within a study.Finally, Clinical Investigator Site Selection Tool (CISST), developed by the Center for Drug Evaluation and Research (CDER), was used to calculate the total risk of each site thereby providing a framework for site selection. The agreement between the predictions of these methods was compared. The overall accuracy and sensitivity of the methods were graphically compared.

Results Spearman’s rank order correlation was used to examine the agreement between the SMART^TM analysis (CluePoints’ software) and the CISST analysis. The average aggregated correlation between the p-values (SMART^TM) and total risk scores (CISST) for all five studies was 0.21, and range from −0.41 to 0.50. The Random Forest models for 2016 and 2017 showed the highest aggregated mean agreement (65.1%) amongst outcomes (NAI, VAI, OAI) for the three available studies. While there does not appear to be a single most accurate approach, the performance of methods under certain circumstances is discussed later in this paper.

Limitations Classifier models based on data mining techniques require historical data (i.e., training data) to develop the model. There is a possibility that sites in the five-summary level datasets were included in the training datasets for the models from the previous year’s research which could result in spurious confirmation of predictive ability. Additionally, the CISST was utilized in three of the five site selection processes, possibly biasing the data.

Conclusion The agreement between methods was lower than expected and no single method emerged as the most accurate.

背景技术在过去的二十年中，临床试验的数量和复杂性急剧增加，大大增加了选择检查部位的难度。FDA的资源有限，因此应谨慎选择地点。

目的确定数据挖掘技术和/或无监督的统计监视是否可以协助确定潜在的临床检查部位。

方法使用FDA已执行或计划进行现场检查的四个新药申请（NDA）和一个生物制剂许可证申请（BLA）的五个摘要级临床现场数据集。研究人员对被检查的地点数量和检查结果视而不见。正在进行的研究项目的前两年（2016-2017）的五个监督学习模型用于预测现场检查结果，即未采取任何行动（NAI），自愿采取行动（VAI）或官方采取行动（ OAI）。由CluePoints（比利时Mont-Saint-Guibert，比利时）开发的无监督统计监视软件应用到研究试验的统计监视（SMART ^TM）软件被用于识别非典型中心（通过p最后，由药物评估和研究中心（CDER）开发的临床研究者选址工具（CISST）用于计算每个地点的总风险，从而为选址提供框架。比较了这些方法的预测之间的一致性。通过图形比较了方法的总体准确性和敏感性。

结果Spearman的等级相关性用于检验SMART ^TM分析（CluePoints的软件）和CISST分析之间的一致性。所有五项研究的p值（SMART ^TM）和总风险评分（CISST）之间的平均聚合相关性为0.21，范围为-0.41至0.50。在三项可用研究的结果（NAI，VAI，OAI）中，2016年和2017年的Random Forest模型显示出最高的平均均值一致性（65.1％）。尽管似乎没有一个最准确的方法，但是本文稍后将讨论在某些情况下这些方法的性能。

局限性基于数据挖掘技术的分类器模型需要历史数据（即训练数据）来开发模型。在前一年研究的模型的训练数据集中，五摘要级别数据集中的站点可能会导致虚假地确认预测能力。此外，在五个站点选择过程中的三个过程中都使用了CISST，这可能会使数据产生偏差。

结论方法之间的一致性低于预期，没有一种方法是最准确的。