Abstract

One of the assessed research elements in the UK’s Research Excellence Framework (REF) exercise in 2014 was the research environment. The quality of the research environment was assessed by expert peer reviewers who were given a set of quantitative factors to support their decision making. However, there is no systematic procedure to integrate this quantitative information into the evaluation process. This article evaluates the relevance of quantitative factors in explaining the assessed quality of the research environment. Findings suggest submitting units with high external research income generation tend to have a better research environment evaluation in almost all the assessed subject areas. The importance given by reviewers to similar quantitative factors was distinctively different in two units of assessment (UoA) in which the evaluation criteria were the same, which highlights the internal inconsistency of the peer review evaluation. Our findings also confirm the existence of the ‘halo effect’ in some UoA where submitting units that belong to the Russell group and have sub-panel members in the REF exercise obtained higher scores even after controlling for the assessed quantitative factors.

中文翻译：

---

研究环境质量的决定因素：2014年英国“卓越研究”框架对环境报告的评估

摘要

研究环境是2014年英国卓越研究框架（REF）中评估的研究内容之一。研究环境的质量由专家级的同行评审人员进行了评估，他们被给予了一系列量化因素以支持其决策。但是，没有系统的程序将此定量信息整合到评估过程中。本文评估了量化因素在解释研究环境评估质量方面的相关性。研究结果表明，提交的具有较高外部研究收入的单位倾向于在几乎所有评估的学科领域中都有更好的研究环境评估。审稿人对相似的量化因素的重视程度在两个评估标准相同的评估单位（UoA）中明显不同，这突显了同行评审评估的内部矛盾。我们的发现还证实了某些UoA中存在“晕轮效应”，即使在控制了评估的量化因素之后，属于Russell组且在REF练习中具有子面板成员的提交单位也获得了更高的分数。