This second special issue of Australian Forestry presents some of the results from ten international forestry research projects that have been funded by the Australian Centre for International Agricultural Research (ACIAR). Since 2005, when the international community agreed to the Paris Declaration on Aid Effectiveness (OECD 2005), there has been a commitment to increase monitoring and evaluation efforts to enable periodic qualitative and quantitative assessments of aid effectiveness. While progress is being made on this commitment, these efforts focus on ten high-level indicators of aid effectiveness, rather than on the effectiveness of particular programs or projects (OECD/UNDP 2016). With forestry research projects, very often there is a long lag time before the desired impact from the research is achieved, which should be taken into account when evaluating effectiveness. The ACIAR currently spends about $115 000 000 annually on agricultural Research for Development (R4D) projects in partner countries (ACIAR 2017), including about $10 000 000 on bilateral forestry research projects that cover many different themes of research and are implemented in many different situations. How then should the effectiveness or success of these investments be evaluated and what do we mean by the terms ‘effectiveness’ and ‘success’? In the development literature ‘effectiveness’ is defined as the extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance (OECD 2002), whereas for ‘success’ there is no agreed definition. Most donor organisations, including ACIAR, commonly use economic impact assessments to estimate the economic benefits that result from such research investments. Certainly this approach can demonstrate impressive returns on investment from selected forest research, as evidenced from ACIAR’s investments in plantation forestry in Indonesia (Lindner 2011) or domestication and improvement of Australian trees in Vietnam (Fisher & Gordon 2007). However, as Lindner et al. (2013) have demonstrated, much of the aggregated economic impacts from ACIAR’s projects come from a small number of highly successful projects. Therefore, it is desirable to consider a range of criteria in determining success of forestry projects. To improve the understanding of what success means for ACIAR forestry projects and what factors affect project success, 90 scientists were interviewed (Bartlett 2018c). They were selected using a purposive strategy because they had worked as project leaders, in-country coordinators or as collaborating scientists on one or more of the 30 selected projects and could still be contacted. From their responses, the following working definition of a successful ACIAR forestry project was developed: a project that uses high quality but flexible scientific methods to achieve planned outputs; enhances the capacity of partners; generates knowledge or technologies that can improve the system under investigation; facilitates ongoing scientific relationships and networks; and results in tangible scientific impacts and benefits for project stakeholders and local communities. This definition recognises that, when judging the success of research projects, there are multiple dimensions to consider and individual projects may only result in partial or incremental improvements to the system being researched as part of a long-term program. While economic impact assessments have a useful role, understanding the cumulative economic benefits arising from research projects does not necessarily help those who fund or implement such projects to understand what causes differential success in research projects. As Stern et al. (2012) have advocated, evaluations should seek to understand what works, what does not work, and why, so that these lessons can be used to replicate, generalise and scale up the results from development interventions. However, there are challenges in trying to compare the results from large numbers of projects that cover many different research themes and that are implemented in many different countries under a wide range of implementation contexts. To help address these challenges and improve the understanding of what enables some research projects to be more successful than others, a new methodology for evaluating the relative success of multiple projects has been developed (Bartlett 2016). In this methodology, two dimensions of project success are considered: the extent to which planned research outputs are achieved and adopted (achievements) and the extent of the impacts resulting from wider adoption, typically outside of the project and beyond its life (impacts). Under each dimension, four different criteria are evaluated, with scores out of ten given for both the achievements and impacts. This approach produces four categories of project success: high achievements-high impacts, high achievements-low impacts, low achievements-low impacts and low achievements-high impacts. This methodology has been applied in three case studies of ACIAR forestry projects, covering ten projects from Vietnam, reported by Bartlett et al. (2017), ten projects from Indonesia, reported by Bartlett (2018a), and ten projects from Papua New Guinea, reported by Bartlett (2018b). About one quarter of the 30 forestry projects (seven) were evaluated as having high achievements and high impacts, with three of these implemented in Vietnam and four in Indonesia. About half (16) of all the projects were evaluated as high achievements and low impacts, with projects in this grouping occurring in all three countries. The remaining one quarter (seven) of the projects had low achievements and low impacts, with two of these occurring in Vietnam, one in Indonesia and the other four in Papua New Guinea. By graphing both of

中文翻译：

---

争取国际林业研究的成功

澳大利亚林业的第二期特刊介绍了由澳大利亚国际农业研究中心 (ACIAR) 资助的十个国际林业研究项目的部分成果。自 2005 年国际社会同意《援助有效性巴黎宣言》（经合组织，2005 年）以来，就一直致力于加强监测和评估工作，以实现对援助有效性的定期定性和定量评估。虽然这一承诺正在取得进展，但这些努力侧重于援助有效性的 10 个高级指标，而不是特定计划或项目的有效性（经合组织/联合国开发计划署 2016）。对于林业研究项目，在实现研究的预期影响之前，通常会有很长的滞后时间，在评估有效性时应考虑到这一点。ACIAR 目前每年在伙伴国家的农业研究促进发展 (R4D) 项目上花费约 115 000 000 美元（ACIAR 2017），其中包括约 10 000 000 美元用于涵盖许多不同研究主题并在许多不同情况下实施的双边林业研究项目. 那么应该如何评估这些投资的有效性或成功？我们所说的“有效性”和“成功”是什么意思？在发展文献中，“有效性”被定义为发展干预目标实现或预期实现的程度，考虑到它们的相对重要性（经合组织 2002 年），而对于“成功”则没有一致的定义。大多数捐助组织，包括 ACIAR，通常使用经济影响评估来估计此类研究投资产生的经济利益。当然，这种方法可以证明从选定的森林研究中获得可观的投资回报，正如 ACIAR 在印度尼西亚的人工林投资（Lindner 2011）或在越南驯化和改良澳大利亚树木（Fisher & Gordon 2007）所证明的。然而，正如林德纳等人。(2013) 已经证明，ACIAR 项目的大部分综合经济影响来自少数非常成功的项目。因此，需要考虑一系列标准来确定林业项目的成功与否。为了加深对 ACIAR 林业项目成功意味着什么以及影响项目成功的因素的理解，对 90 位科学家进行了采访（Bartlett 2018c）。他们是通过有目的的策略被选中的，因为他们曾担任过 30 个选定项目中的一个或多个项目的项目负责人、国内协调员或合作科学家，并且仍然可以联系到他们。根据他们的回答，制定了成功的 ACIAR 林业项目的以下工作定义：使用高质量但灵活的科学方法实现计划产出的项目；提高合作伙伴的能力；产生可以改进被调查系统的知识或技术；促进持续的科学关系和网络；并为项目利益相关者和当地社区带来切实的科学影响和利益。这个定义承认，在判断研究项目的成功时，有多个方面需要考虑，作为长期计划的一部分，个别项目可能只会对正在研究的系统进行部分或增量改进。尽管经济影响评估具有有用的作用，但了解研究项目产生的累积经济效益并不一定有助于资助或实施此类项目的人了解导致研究项目取得不同成功的原因。正如斯特恩等人。(2012) 主张，评估应设法了解什么有效，什么无效，以及为什么，以便这些经验教训可用于复制、推广和扩大发展干预措施的结果。然而，试图比较涵盖许多不同研究主题并在许多不同国家在广泛实施背景下实施的大量项目的结果存在挑战。为了帮助应对这些挑战并加深对是什么使某些研究项目比其他项目更成功的理解，开发了一种评估多个项目相对成功的新方法（Bartlett 2016）。在该方法论中，考虑了项目成功的两个方面：实现和采用计划研究成果的程度（成就）以及更广泛采用所产生的影响程度，通常是在项目之外和生命周期之外（影响）。在每个维度下，评估四个不同的标准，对成就和影响都给予满分十分。这种方法产生四类项目成功：高成就-高影响、高成就-低影响、低成就-低影响和低成就-高影响。Bartlett 等人报道，该方法已应用于 ACIAR 林业项目的三个案例研究，涵盖来自越南的十个项目。（2017 年）、Bartlett 报告的 10 个印度尼西亚项目（2018a）和 Bartlett 报告的巴布亚新几内亚的 10 个项目（2018b）。在 30 个林业项目（7 个）中，约有四分之一被评估为成就高、影响大，其中 3 个在越南实施，4 个在印度尼西亚实施。所有项目中约有一半 (16) 被评为高成就和低影响，该组中的项目发生在所有三个国家。其余四分之一（七个）项目成果低、影响小，其中两个发生在越南，一个发生在印度尼西亚，另外四个发生在巴布亚新几内亚。通过绘制两个