Agronomists have increasingly conducted experiments on-farm, in an attempt to increase the wider applicability (external validity) of their experimental findings and their relevance for agricultural development. This review assesses the way in which on-farm experimental studies address the scope or generalisability of their findings when based on a limited number of farms. A central question is how on-farm studies define the environment or research population in which the on-farm trial findings are valid, or are valuable for. Such an assessment is, of course, conditional on the (internal) validity of the experimental findings. We therefore first analyse how authors of on-farm experimental studies describe the factors that may shape experimental outcomes. As agronomic experiments often use ‘yield’ as dependent variable to assess treatment effects, we developed a procedure to score studies on their descriptions of yield-determining factors. Although experimental validity principally rests upon the reproducibility of the experiment and its findings, we found that on the basis of the information provided in published on-farm experimental studies, it is often difficult or impossible to reproduce the experimental design. Nutrient management, weed management and crop information are best described, whereas land preparation, field history and management of pests and water are rarely described. Further, on-farm experimental studies often compare treatments to a ‘farmer practice’ reference or control treatment which is assumed to be widely and uniformly practiced and known to the reader. The wider applicability or external validity is often poorly addressed in the reviewed studies. Most do not explicitly define the research population and/or environment in which (they expect) the experimental findings to work. Academic textbooks on agronomic experimentation are remarkably silent on both the internal and external validity of on-farm experimentation. We therefore argue for more systematic investigations and descriptions of the research population and settings to which on-farm experimental studies seek to generalise their findings.

中文翻译：

---

非洲农场实验研究的可重复性和外部有效性

农学家越来越多地在农场进行实验，以试图增加其实验结果的更广泛适用性（外部有效性）及其与农业发展的相关性。本综述评估了农场实验研究在基于有限数量的农场时解决其研究结果的范围或普遍性的方式。一个核心问题是农场研究如何定义农场试验结果有效或有价值的环境或研究人群。当然，这样的评估取决于实验结果的（内部）有效性。因此，我们首先分析农场实验研究的作者如何描述可能影响实验结果的因素。由于农艺实验经常使用“产量”作为因变量来评估处理效果，我们开发了一个程序来对他们对产量决定因素的描述的研究进行评分。虽然实验有效性主要取决于实验的可重复性及其结果，但我们发现，根据已发表的农场实验研究中提供的信息，重现实验设计通常很困难或不可能。营养管理、杂草管理和作物信息得到了最好的描述，而很少描述土地整备、田间历史以及害虫和水的管理。此外，农场实验研究通常将处理与“农民实践”参考或对照处理进行比较，后者被假定为广泛且统一地实践并为读者所知。在审查的研究中，更广泛的适用性或外部有效性通常没有得到很好的解决。大多数人没有明确定义（他们期望）实验结果起作用的研究人群和/或环境。关于农艺实验的学术教科书对农场实验的内部和外部有效性都非常沉默。因此，我们主张对农场实验研究试图概括其发现的研究人群和环境进行更系统的调查和描述。