Background

Success shapes the lives and careers of scientists. But success in science is difficult to define, let alone to translate in indicators that can be used for assessment. In the past few years, several groups expressed their dissatisfaction with the indicators currently used for assessing researchers. But given the lack of agreement on what should constitute success in science, most propositions remain unanswered. This paper aims to complement our understanding of success in science and to document areas of tension and conflict in research assessments.

Methods

We conducted semi-structured interviews and focus groups with policy makers, funders, institution leaders, editors or publishers, research integrity office members, research integrity community members, laboratory technicians, researchers, research students, and former-researchers who changed career to inquire on the topics of success, integrity, and responsibilities in science. We used the Flemish biomedical landscape as a baseline to be able to grasp the views of interacting and complementary actors in a system setting.

Results

Given the breadth of our results, we divided our findings in a two-paper series, with the current paper focusing on what defines and determines success in science. Respondents depicted success as a multi-factorial, context-dependent, and mutable construct. Success appeared to be an interaction between characteristics from the researcher (Who), research outputs (What), processes (How), and luck. Interviewees noted that current research assessments overvalued outputs but largely ignored the processes deemed essential for research quality and integrity. Interviewees suggested that science needs a diversity of indicators that are transparent, robust, and valid, and that also allow a balanced and diverse view of success; that assessment of scientists should not blindly depend on metrics but also value human input; and that quality should be valued over quantity.

Conclusions

The objective of research assessments may be to encourage good researchers, to benefit society, or simply to advance science. Yet we show that current assessments fall short on each of these objectives. Open and transparent inter-actor dialogue is needed to understand what research assessments aim for and how they can best achieve their objective.

Study Registration

osf.io/33v3m.

中文翻译：

---

重新思考研究中的成功，诚信和文化（第1部分）-关于科学成功的多因素定性研究

背景

成功决定着科学家的生活和职业。但是很难确定科学上的成功，更不用说转化为可用于评估的指标了。在过去的几年中，几个小组对当前用于评估研究人员的指标表示不满。但是，由于对于在科学上应如何构成成功缺乏共识，因此大多数命题仍未得到解答。本文旨在补充我们对科学成功的理解，并记录研究评估中的紧张和冲突领域。

方法

我们与决策者，出资者，机构负责人，编辑或出版商，研究诚信办公室成员，研究诚信社区成员，实验室技术人员，研究人员，研究学生以及改变职业以进行咨询的前研究人员进行了半结构化访谈和焦点小组讨论。科学中的成功，正直和责任等主题。我们使用佛兰德生物医学景观作为基准，以便能够掌握系统设置中相互作用和互补的参与者的观点。

结果

鉴于研究结果的广度，我们将研究结果分为两篇论文系列，而当前论文的重点是定义和确定科学成功的因素。受访者将成功描述为一种多因素，依赖于上下文且易变的结构。成功似乎是研究人员（谁），研究成果（什么），过程（如何）和运气之间的相互作用。受访者指出，当前的研究评估高估了产出，但在很大程度上忽略了对研究质量和完整性必不可少的过程。受访者认为，科学需要透明，健壮和有效的多种指标，同时也要有一个平衡而多样的成功观。科学家的评估不应盲目地依赖指标，而应重视人类的投入；

结论

研究评估的目的可能是鼓励优秀的研究人员，造福社会或只是促进科学发展。然而，我们表明，当前的评估未能实现这些目标。需要进行公开透明的参与者间对话，以了解研究评估的目的以及如何最好地实现其目标。

研究注册

osf.io/33v3m。