BACKGROUND Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity. METHODS We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes. RESULTS A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively. CONCLUSIONS Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.

中文翻译：

---

在试验性和有效性/有效性试验中识别和评估概化性偏倚的风险：系统评价和荟萃分析。

背景技术对行为干预的初步评估（称为先导研究）早于许多大型功效/功效试验的开展。试点研究告知功效/功效试验的能力取决于对试验结果的设计，交付和解释的仔细考虑，以避免夸大的早期发现，而这可能导致随后的功效/功效试验失败。试点研究中的“概论偏差（RGB）风险”可能会降低在较大的功效/功效试验中重复结果的可能性。我们旨在生成潜在RGB的操作列表，并在已发表的关于儿童肥胖的先导研究和更大，功能更强大的试验对中评估它们的影响。方法我们进行了系统的文献综述，以鉴定已发表的先导研究，该研究已发表了相同或相似干预措施的大规模试验。搜索已更新并在2018年12月31日前完成。符合条件的研究是涉及与儿童肥胖有关的主题（例如，预防/治疗，减轻体重，体育锻炼，饮食，睡眠，筛查时间/久坐行为）。提取的信息包括研究特征和所有结果。定义并编码了9种RGB的列表：干预强度偏差，实施支持偏差，递送媒介偏差，目标受众偏差，持续时间偏差，设置偏差，测量偏差，方向性结论偏差和结果偏差。三位审阅者针对RGB的存在进行了独立编码。进行了多级随机效应荟萃分析，以研究偏倚与研究结果的关联。结果总共鉴定出39对试验和较大试验对。偏见的频率各不相同：递送媒介偏见（19/39对），持续时间偏见（15/39），实施支持偏见（13/39），结果偏见（6/39），测量偏见（4/39），定向结论偏差（3/39），目标受众偏差（3/39），干预强度偏差（1/39）和设置偏差（0/39）。在荟萃分析中，传递剂，实施支持，持续时间和测量偏差与-0.325（95CI-0.556至-0.094），-0.346（-0.640至-0.052），-0.342（ -0.498至-0.187）和-0.360（-0.631至-0.089）。