Background

Mentoring programs pair non-familial adults with children and adolescents for the purposes of promoting positive youth development. Although these programs are widely popular, evaluations tend to show that mentoring programs have, on average, modest effects on youth outcomes. Some researchers have suggested that mentoring programs should homogenize mentoring activities as a means for increasing effect sizes of programs. :

Objective

This paper describes why heterogeneity of mentoring activities should not necessarily be regarded as a problem (i.e., a bug) that needs correction; rather it is more representative of the construct of mentoring as it is popularly understood and also desirable because of the potential to improve access and quality of prevention services (i.e., it is a feature).

Method

We present different simulated scenarios demonstrating how evaluations of mentoring programs may change the estimates of treatment effects depending on how evaluators measure programmatic activities and approach analyses.

Results

Analyses illustrate that commonly used evaluation strategies that treatment effects may be underestimated when mentoring activities are not measured and are paired with common analytic approaches (e.g., intent-to-treat analyses). Simulated scenarios also highlight alternative approaches for defining programmatic elements and evaluating programs to produce a more robust estimate of effects.

Conclusions

The optimal strategy for evaluating mentoring services depends on the particular features of the program as well as the goals of the evaluation. One approach researchers might take is to evaluate specific mentoring practices, before evaluating mentoring programs, to begin to understand program impact.

中文翻译：

---

这不是错误，而是一个功能：通过各种活动评估指导计划

背景

辅导计划将非家庭成人与儿童和青少年配对，以促进积极的青年发展。尽管这些计划广受欢迎，但评估往往表明，指导计划平均对青年成果有中等程度的影响。一些研究人员建议，指导计划应使指导活动同质化，以增加计划效果的大小。：

客观的

本文描述了为什么导师活动的异质性不应该被认为是需要纠正的问题（例如bug）；相反，它更能代表指导的结构，因为它有可能改善预防服务的获取和质量（即这是一个功能），因此已广为人知，并且也是可取的。

方法

我们提出了不同的模拟方案，这些方案演示了指导方案的评估如何根据评估者对计划活动和方法分析的评估方式来改变对治疗效果的估计。

结果

分析表明，常用的评估策略是：在未衡量指导活动的情况下，可能会低估治疗效果，并与常见的分析方法（例如，意向治疗分析）结合使用。模拟场景还重点介绍了定义程序元素和评估程序以产生更可靠的效果估计的替代方法。

结论

评估指导服务的最佳策略取决于计划的特定功能以及评估的目标。研究人员可能采取的一种方法是，在评估指导计划之前，先评估特定的指导实践，以开始了解计划的影响。