-
Practicing responsible research assessment: Qualitative study of faculty hiring, promotion, and tenure assessments in the United States Research Evaluation (IF 2.8) Pub Date : 2024-03-14 Alexander Rushforth, Sarah De Rijcke
Recent times have seen the growth in the number and scope of interacting professional reform movements in science, centered on themes such as open research, research integrity, responsible research assessment, and responsible metrics. The responsible metrics movement identifies the growing influence of quantitative performance indicators as a major problem and seeks to steer and improve practices around
-
Organizational changes and research performance: A multidimensional assessment Research Evaluation (IF 2.8) Pub Date : 2024-02-17 José Luis Jiménez-Andrade, Ricardo Arencibia-Jorge, Miguel Robles-Pérez, Julia Tagüeña, Tzipe Govezensky, Humberto Carrillo-Calvet, Rafael A Barrio, Kimmo Kaski
This paper analyzes the research performance evolution of a scientific institute, from its genesis through various stages of development. The main aim is to obtain, and visually represent, bibliometric evidence of the correlation of organizational changes on the development of its scientific performance; particularly, structural and leadership changes. The study involves six bibliometric indicators
-
What is a high-quality research environment? Evidence from the UK’s research excellence framework Research Evaluation (IF 2.8) Pub Date : 2024-02-16 Matthew Inglis, Elizabeth Gadd, Elizabeth Stokoe
As part of the UK university sector’s performance-related research funding model, the ‘REF’ (Research Excellence Framework), each discipline-derived ‘Unit of Assessment’ must submit a statement to provide information about their environment, culture, and strategy for enabling research and impact. Our aim in this paper is to identify the topics on which these statements focus, and how topic variation
-
Towards a sustainable and responsible model for monitoring open science and research—analysis of the Finnish model for monitoring open science and research Research Evaluation (IF 2.8) Pub Date : 2024-02-16 Laura Himanen, Susanna Nykyri
Advancing and supporting the development of open science practices is of global interest, and subsequently also the monitoring of the ongoing developments is more and more on the agenda. In Finland, monitoring open science has a relatively long history, and the first evaluation of research performing and research funding organisations took place already in 2015. The system has evolved over the years
-
Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review Research Evaluation (IF 2.8) Pub Date : 2024-02-09 Tony Ross-Hellauer, Serge P J M Horbach
Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain. This scoping review updates previous
-
Effective mission-oriented research: A new framework for systemic research impact assessment Research Evaluation (IF 2.8) Pub Date : 2024-02-09 Lena Pfeifer, Katharina Helming
Mission-oriented research combines a wide array of natural and social science disciplines to offer solutions for complex and multi-dimensional challenges such as climate change, loss of biodiversity, and scarcity of natural resources. The utilization of the outputs of mission-oriented research aims for changes in behavior, policy and practice resulting in real world impacts. Systematically assessing
-
The legal foundation of responsible research assessment: An overview on European Union and Italy Research Evaluation (IF 2.8) Pub Date : 2023-11-29 Ginevra Peruginelli, Janne Pölönen
During the past decade, responsible research assessment (RRA) has become a major science policy goal to advance responsible research and innovation and open science. Starting with the DORA declaration in 2012, common understanding of the needs and demands of RRA has been shaped by a growing number of initiatives, culminating in a European Commission supported Agreement on reforming research assessment
-
The conflict of impact for early career researchers planning for a future in the academy Research Evaluation (IF 2.8) Pub Date : 2023-11-22 Marta Natalia Wróblewska, Corina Balaban, Gemma Derrick, Paul Benneworth
It has been argued that due to the growing importance attributed to research impact and forms of its evaluation, an academic ‘culture of impact’ is emerging. It would include certain concepts, values, and skills related to the area of generating and documenting impact. We use thematic and discourse analysis to analyse open answers from 100 questionnaires on research impact submitted by ECRs working
-
Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices Research Evaluation (IF 2.8) Pub Date : 2023-11-07 Gemma Elizabeth Derrick, Alessandra Zimmermann, Helen Greaves, Jonathan Best, Richard Klavans
Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the ‘best’ decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants
-
Effects of transdisciplinary research on scientific knowledge and reflexivity Research Evaluation (IF 2.8) Pub Date : 2023-11-06 Oskar Marg, Lena Theiler
Transdisciplinary research (TDR) is conceptualized as not only providing societal effects but also benefiting academia. However, recent literature on the evaluation of TDR has focused almost entirely on the societal effects of TDR. A discussion of the scientific effects of TDR is needed to do justice to the potential of this research mode. To date, little empirical research has focused on the effects
-
Theory of systems change: An initial, middle-range theory of public health research impact Research Evaluation (IF 2.8) Pub Date : 2023-10-19 Melinda Craike, Bojana Klepac, Amy Mowle, Therese Riley
There is increasing attention on evidencing research impact and applying a systems thinking perspective in public health. However, there is limited understanding of the extent to which and how public health research that applies a systems thinking perspective contributes to changes in system behaviour and improved population health outcomes. This paper addresses the theoretical limitations of research
-
The effect of applied research institutes on invention: Evidence from the Fraunhofer centres in Europe Research Evaluation (IF 2.8) Pub Date : 2023-10-13 Pedro Llanos-Paredes
This study examines the impact of the Fraunhofer Society, Europe’s largest network of applied research institutes, on patent applications. A difference-in-differences strategy was employed exploiting the establishment of five new Fraunhofer centres in the 2000s. The panel includes 65,963 European applicants (both firms and independent inventors) between 1980 and 2019. The results show that establishing
-
Warnings of declining research productivity: Does Italy buck the trend? Research Evaluation (IF 2.8) Pub Date : 2023-09-17 Giovanni Abramo, Ciriaco Andrea D’Angelo
The paper takes a scientometric approach to measure the change in research productivity of Italian academics before the COVID-19 pandemic outbreak. We propose a composite output/input bibliometric indicator and apply it at the field level, conducting a longitudinal analysis. Although the number of academics in the national academic system has decreased, we register very strong growth in both the number
-
Do funding modes matter? A multilevel analysis of funding allocation mechanisms on university research performance Research Evaluation (IF 2.8) Pub Date : 2023-09-04 Thomas Zacharewicz, Noemi Pulido Pavón, Luis Antonio Palma Martos, Benedetto Lepori
Over the last decades, most EU countries have profoundly reshaped their public research funding systems by shifting from traditional institutional block-funding towards more project-based mechanisms. The main rationale underlying this evolution builds on the assumption that project funding would foster research performance through the introduction of competitive allocation mechanisms. In contrast with
-
Methods for measuring social and conceptual dimensions of convergence science Research Evaluation (IF 2.8) Pub Date : 2023-07-20 Alexander Michael Petersen, Felber Arroyave, Ioannis Pavlidis
Convergence science is an intrepid form of interdisciplinarity defined by the US National Research Council as ‘the coming together of insights and approaches from originally distinct fields’ to strategically address grand challenges. Despite its increasing relevance to science policy and institutional design, there is still no practical framework for measuring convergence. We address this gap by developing
-
How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework Research Evaluation (IF 2.8) Pub Date : 2023-07-14 Ohid Yaqub, Dmitry Malkov, Josh Siepel
Although ex post evaluation of impact is increasingly common, the extent to which research impacts emerge largely as anticipated by researchers, or as the result of serendipitous and unpredictable processes, is not well understood. In this article, we explore whether predictions of impact made at the funding stage align with realized impact, using data from the UK’s Research Excellence Framework (REF)
-
Turning academics into researchers: The development of National Researcher Categorization Systems in Latin America Research Evaluation (IF 2.8) Pub Date : 2023-07-13 Federico Vasen, Nerina F Sarthou, Silvina A Romano, Brenda D Gutiérrez, Manuel Pintos
Evaluation procedures play a crucial role in science and technology systems, particularly within academic career structures. This article focuses on an approach to evaluation that has gained prominence in Latin America over the past four decades. This scheme assesses the individual performance of academics based on their academic activities and outputs and assigns them a ‘researcher category’, which
-
Can journal reviewers dependably assess rigour, significance, and originality in theoretical papers? Evidence from physics Research Evaluation (IF 2.8) Pub Date : 2023-06-30 Mike Thelwall, Janusz A Hołyst
Peer review is a key gatekeeper for academic journals, attempting to block inadequate submissions or correcting them to a publishable standard, as well as improving those that are already satisfactory. The three key aspects of research quality are rigour, significance, and originality but no prior study has assessed whether journal reviewers are ever able to judge these effectively. In response, this
-
The forms of societal interaction in the social sciences, humanities and arts: Below the tip of the iceberg Research Evaluation (IF 2.8) Pub Date : 2023-06-02 Elea Giménez-Toledo, Julia Olmos-Peñuela, Elena Castro-Martínez, François Perruchas
Science policymakers are devoting increasing attention to enhancing the social valorization of scientific knowledge. Since 2010, several international evaluation initiatives have been implemented to assess knowledge transfer and exchange practices and the societal impacts of research. Analysis of these initiatives would allow investigation of the different knowledge transfer and exchange channels and
-
Boundary-work and social closure in academic recruitment: Insights from the transdisciplinary subject area Swedish as a Second Language Research Evaluation (IF 2.8) Pub Date : 2023-05-15 Natalia Ganuza, Linus Salö
This article explores practices of evaluation in academic recruitment in Swedish as a Second Language (SSL), an expanding and transdisciplinary subject area. As is common elsewhere, Swedish academia relies on a tradition of external expert review intended to ensure a meritocratic process. Here, we present an analysis of 109 written expert reports concerning recruitment to 57 positions in SSL during
-
The missing links of research impact Research Evaluation (IF 2.8) Pub Date : 2023-05-12 David Budtz Pedersen, Rolf Hvidtfeldt
In this article, we present a conceptual framework for studying research impact focusing on the foundations that need to be in place to accelerate an observable change of policy, practice or behaviour. The article investigates the relationship between micro-impacts and societal change, and how smaller impacts scale into larger cascades of end effects and value creation. We define micro-impacts as interactions
-
Promoting narrative CVs to improve research evaluation? A review of opinion pieces and experiments Research Evaluation (IF 2.8) Pub Date : 2023-04-19 Frédérique Bordignon, Lauranne Chaignon, Daniel Egret
As the academic community has become increasingly concerned about the drifts of research evaluation, mostly researchers’ evaluation, because of the overreliance on metrics, many expert groups have made recommendations to improve the way researchers should be evaluated. In this study, we focus on the recommendation to use narrative curriculum vitae (CVs). We review 28 opinion pieces and 7 experiments
-
The price of quality: Scholarly publishing business is the primary predictor of citation-based indicators of journal performance in ecology and evolutionary biology Research Evaluation (IF 2.8) Pub Date : 2023-04-19 Adriana Ruggiero
Citation-based indicators of journals’ performance are often assumed to offer an objective, albeit indirect, way of measuring research quality. However, recent concerns about their applicability for research evaluation suggested these indicators could depend on historical and socioeconomic factors associated with scholarly publishing tradition and business, respectively. The present study addressed
-
Improving universities’ activities in academic startup support through public interventions: The effectiveness of the German programme ‘EXIST—leverage of potentials’ Research Evaluation (IF 2.8) Pub Date : 2023-03-29 Christoph E Mueller
Academic startups have a considerable economic impact, which is why public support programmes for them are considered an important component of innovation and technology policy. In this context, university support programmes can be an important part of the policy toolkit by aiming to improve universities’ startup environment and thus promote startup activity at those institutions. Assessing the effectiveness
-
Changing conceptualization of innovation in the European Union and its impact on universities: Critical junctures and evolving institutional demands Research Evaluation (IF 2.8) Pub Date : 2023-03-17 Ridvan Cinar, Paul Benneworth, Lars Coenen
This article explores underlying mechanisms triggering a change in conceptualization of innovation in the European Union (EU), the impact of this change on institutional demands upon European universities and implications for evaluation procedures. We mobilize the theoretical concept of critical junctures to explore significant periods that have affected understanding of innovation in the EU as well
-
Gender diversity and publication activity—an analysis of STEM in the UK Research Evaluation (IF 2.8) Pub Date : 2023-03-09 Yasaman Sarabi, Matthew Smith
Gender diversity in STEM remains a significant issue, as the field continues to be a male dominated one, despite increased attention on the subject. This article examines the interplay between gender diversity on projects funded by a major UK research council, the Engineering and Physical Sciences Research Council, and the publication activity of a project, as measured by the average journal quality
-
Early career academic's odyssey: A narrative study of her professional identity construction Research Evaluation (IF 2.8) Pub Date : 2023-03-08 Javier Mula-Falcón, Katia Caballero
Recent decades have seen a major transformation of the Spanish university system caused by changes introduced in the teaching staff evaluation procedure in which research has been prioritized. As a result, there has been a growing interest in studying how these procedures impact the way in which early career academics construct their professional identity. In this vein, this article aims to analyze
-
Assessing the variety of collaborative practices in translational research: An analysis of scientists’ ego-networks Research Evaluation (IF 2.8) Pub Date : 2023-03-08 Adrián A Díaz-Faes, Oscar Llopis, Pablo D’Este, Jordi Molas-Gallart
Translational research policies aim to reshape how biomedical scientists organize, conceive, and conduct science in order to accelerate healthcare improvements and medical innovations. Yet most analyses and evaluations of these initiatives focus on measuring the outputs generated in the different stages of the research process rather than observing scientists’ research practices directly. In this article
-
On the societal impact of publicly funded Circular Bioeconomy research in Europe Research Evaluation (IF 2.8) Pub Date : 2023-02-23 Ana Sofia Brandão, José M R C A Santos
Europe has taken a world leadership position in setting policy priorities for Circular Bioeconomy (CBE) as a key determinant of economic, social, and environmental sustainability. Consequently, European R&D investment in this area keeps growing along with the societal pressure to demonstrate the return of investment of publicly funded projects. Thus, this work presents a pioneering exploratory analysis
-
The rocky road to translational science: An analysis of Clinical and Translational Science Awards Research Evaluation (IF 2.8) Pub Date : 2023-01-19 Yeon Hak Kim, You-Na Lee, Seokkyun Woo
Studies point out that the productivity decline in biomedicine is in significant part due to difficulties in translating basic science into clinical application. To promote translational research, the US NIH launched the Clinical and Translational Science Awards (CTSA) program in 2006. Prior evaluations of the CTSA program often assumed that the key to translation is inter-organizational collaboration
-
Describing the state of a research network: A mixed methods approach to network evaluation. Research Evaluation (IF 2.8) Pub Date : 2022-10-28 James M Bowen,Mathieu Ouimet,Justin Lawarée,Joanna Bielecki,Ashley Rhéaume,Caylee Greenberg,Valeria E Rac
Diabetes Action Canada Strategy for Patient-Oriented Research (SPOR) Network in Chronic Disease was formed in 2016 and is funded primarily through the Canadian Institutes of Health Research (CIHR). We propose a novel mixed-methods approach to a network evaluation integrating the State of Network Evaluation framework and the Canadian Academy of Health Sciences (CAHS) preferred framework and indicators
-
Explaining employment sector choices of doctoral graduates in Germany Research Evaluation (IF 2.8) Pub Date : 2022-10-03 Lea Goldan, Steffen Jaksztat, Christiane Gross
Previous research in different national contexts has shown that individual preferences for certain job attributes, objective labour market conditions, subjective career prospects, and external encouragement shape doctoral graduates’ career decisions. For Germany, where the number of awarded doctoral degrees is highest within the European Union and where no established academic tenure-track system exists
-
Research governance and the dynamics of science: A framework for the study of governance effects on research fields Research Evaluation (IF 2.8) Pub Date : 2022-08-19 Maria Nedeva, Mayra M Tirado, Duncan A Thomas
This article offers a framework for the study of research governance effects on scientific fields framed by notions of research quality and the epistemic, organizational, and career choices they entail. The framework interprets the contested idea of ‘quality’ as an interplay involving notion origins, quality attributes, and contextual sites. We mobilize the origin and site components, to frame organizational-level
-
The footprint of a metrics-based research evaluation system on Spain’s philosophical scholarship: An analysis of researchers’ perceptions Research Evaluation (IF 2.8) Pub Date : 2022-08-12 Ramón A Feenstra, Emilio Delgado López-Cózar
The use of bibliometric indicators in research evaluation has a series of complex impacts on academic inquiry. These systems have gradually spread into a wide range of locations and disciplines, including the humanities. The aim of this study is to examine their effects as perceived by philosophy and ethics researchers in Spain, a country where bibliometric indicators have long been used to evaluate
-
The impact of Italian performance-based research funding systems on the intensity of international research collaboration Research Evaluation (IF 2.8) Pub Date : 2022-07-29 Giovanni Abramo, Ciriaco Andrea D’Angelo
The study of national research assessment exercises serves to evaluate the effectiveness of policies versus their objectives and to improve the formulation of future initiatives. The aim of the current study is to verify whether the introduction of the first performance-based research funding in Italy, based on the 2004–10 VQR assessment, achieved the intended objective of inducing greater international
-
Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations Research Evaluation (IF 2.8) Pub Date : 2022-07-18 João M Santos
Securing research funding is essential for all researchers. The standard evaluation method for competitive grants is through evaluation by a panel of experts. However, the literature notes that peer review has inherent flaws and is subject to biases, which can arise from differing interpretations of the criteria, the impossibility for a group of reviewers to be experts in all possible topics within
-
Evaluation of the arts in performance-based research funding systems: An international perspective Research Evaluation (IF 2.8) Pub Date : 2022-07-18 Kamila Lewandowska, Emanuel Kulczycki, Michael Ochsner
This article provides a comprehensive analysis of the evaluation of the arts within performance-based research funding systems (PRFSs). Previous literature on PRFSs has overlooked the arts and focussed primarily on outputs in relation to the sciences and humanities. We develop a typology of how artistic outputs are evaluated within 10 countries’ PRFSs, operating in Australia, the Czech Republic, Italy
-
Valorization of transdisciplinary research: An evaluation approach and empirical illustration Research Evaluation (IF 2.8) Pub Date : 2022-06-30 Stefania Munaretto, Caro E Mooren, Laurens K Hessels
In recent times, there has been a surge of impact-oriented, transdisciplinary research programmes and projects integrating multiple disciplines, types of knowledge and practices. An essential element often mentioned in the literature to improve the performance of these programmes and support impact delivery is continuous reflection and learning via evaluation. We argue that because a standard format
-
Under pressure: The extent and distribution of perceived pressure among scientists in Germany, Austria, and Switzerland Research Evaluation (IF 2.8) Pub Date : 2022-06-21 David Johann, Isabel J Raabe, Heiko Rauhut
While it has been stressed repeatedly that academics nowadays have come to face extensive pressure, the extent and distribution of pressure to publish and to secure third-party funding has not been systematically investigated on a large scale. Based on the Zurich Survey of Academics, a representative large-scale web survey among academics working at universities in Germany, Austria, and Switzerland
-
University policy engagement bodies in the UK and the variable meanings of and approaches to impact Research Evaluation (IF 2.8) Pub Date : 2022-06-16 Hannah Durrant, Eleanor MacKillop
Over the last decade, there has been a proliferation of policy engagement bodies set up within universities worldwide. The present study focuses on the British experience of this phenomenon but with relevance to other contexts. Multiple factors are at play to explain this growth, from the Research Excellence Framework impact agenda (which assesses and ranks the quality of research in UK universities
-
Automated citation recommendation tools encourage questionable citations Research Evaluation (IF 2.8) Pub Date : 2022-06-02 Serge P J M Horbach, Freek J W Oude Maatman, Willem Halffman, Wytske M Hepkema
Citing practices have long been at the heart of scientific reporting, playing both socially and epistemically important functions in science. While such practices have been relatively stable over time, recent attempts to develop automated citation recommendation tools have the potential to drastically impact citing practices. We claim that, even though such tools may come with tempting advantages,
-
A participatory approach to tracking system transformation in clusters and innovation ecosystems—Evolving practice in Sweden’s Vinnväxt programme Research Evaluation (IF 2.8) Pub Date : 2022-04-01 Emily Wise,Moa Eklund,Madeline Smith,James Wilson
Abstract For decades, cluster initiatives and funding programmes have been used as instruments of industrial and innovation policy—addressing system failures by strengthening linkages among actors, fostering innovation, and developing more effective innovation systems. More recently, a growing segment of these initiatives are also focused on driving system-level transformation and contributing to broader
-
Interdisciplinary research and policy impacts: Assessing the significance of knowledge coproduction Research Evaluation (IF 2.8) Pub Date : 2022-03-25 Luisa Veras de Sandes-Guimarães, Raquel Velho, Guilherme Ary Plonski
Research impact on various societal spheres has been increasingly demanded for funding purposes and as a form of demonstrating the relevance of scientific research for societal problems. In this context, interest in interdisciplinary research and knowledge coproduction has grown as a path to achieve this expected impact, but few studies have considered its conjoint association with societal impact
-
Research calls, competition for funding and inefficiency Research Evaluation (IF 2.8) Pub Date : 2022-03-24 António Osório,Lutz Bornmann
Abstract Research groups spend time and resources in the process of applying for funding. This issue raises important questions regarding inefficiency and whether the currently used funding mechanisms are adequate. This article aims to identify ways of reducing the inefficiency and the waste of resources when making research funding calls. We look at four ways of reducing inefficiency. Inefficiency
-
Predicting funded research project performance based on machine learning Research Evaluation (IF 2.8) Pub Date : 2022-03-15 Hoon Jang
Abstract Increasing investment and interest in research and development (R&D) requires an efficient management system for achieving better research project outputs. In tandem with this trend, there is a growing need to develop a method for predicting research project outputs. Motivated by this, using information gathered in the early stage of projects, this study addresses the problem of predicting
-
Does R&D tax credit impact firm behaviour? Micro evidence for Portugal Research Evaluation (IF 2.8) Pub Date : 2022-01-27 Alexandre Paredes, Joana Mendonça, Fernando Bação, Bruno Damásio
In this study, we use panel data to analyse the impact of an R&D tax credit on R&D personnel, particularly the impact on Ph.D. holders allocation, comparing low R&D intensity firms with medium-high and high R&D intensity firms. The results show that, in medium-high and high R&D intensity firms, the R&D tax credit had a significant impact on allocating Ph.D. holders in firms after 3 years of participation
-
Evaluating the Revised National Institutes of Health clinical trial definition impact on recruitment progress Research Evaluation (IF 2.8) Pub Date : 2022-01-26 Eugene I Kane, Gail L Daumit, Kevin M Fain, Roberta W Scherer, Emma Elizabeth McGinty
Background The National Institutes of Health (NIH) announced a revised, expanded definition of ‘clinical trial’ in 2014 to improve trial identification and administrative compliance. Some stakeholders voiced concerns that the policy added administrative burden potentially slowing research progress. Methods This quasi-experimental study examined the difference-in-differences impact of the new NIH clinical
-
Does monitoring performance act as an incentive for improving research performance? National and organizational level analysis of Finnish universities Research Evaluation (IF 2.8) Pub Date : 2022-01-25 Laura Himanen, Hanna-Mari Puuska
Monitoring of research performance, especially performance-based allocation of research funding inevitably creates intended and unintended incentives for universities. In this article, we study if monitoring performance acts as an incentive for improved research performance by scrutinizing the development of two essential indicators of Finnish universities’ research performance, publication output
-
The epistemic, production, and accountability prospects of social impact: An analysis of strategic research proposals Research Evaluation (IF 2.8) Pub Date : 2022-01-10 Juha-Pekka Lauronen
This article addresses the debate on pre-evaluative choices of impact depictions and the forms of responses between applicants and funders. By adopting a reflexive perspective on the social impact of social sciences, this article explores researchers’ vocabularies in the research proposals and mid-term reports of consortiums during the Strategic Research Council (SRC) calls in the period 2015–18. This
-
Societal targeting in researcher funding: An exploratory approach Research Evaluation (IF 2.8) Pub Date : 2021-12-26 Irene Ramos-Vielba, Duncan A Thomas, Kaare Aagaard
Shaping public research to enhance its societal contribution has become a key policy concern. Against this background, how research funding may stimulate the societal orientation of scientific research—or how funding is societally targeted—has been underexplored. This article proposes an exploratory approach to characterize societal targeting in individual researcher funding, based on four key societal
-
Competitive exposure and existential recognition: Visibility and legitimacy on academic social networking sites Research Evaluation (IF 2.8) Pub Date : 2021-12-18 Helena Francke, Björn Hammarfelt
Over the past decade, academic social networking sites, such as ResearchGate and Academia.edu, have become a common tool in academia for accessing publications and displaying metrics for research evaluation and self-monitoring. In this conceptual article, we discuss how these academic social networking sites, as devices of evaluation that build on both traditional values, objects, and metrics in academic
-
Assessing research excellence: Evaluating the Research Excellence Framework Research Evaluation (IF 2.8) Pub Date : 2021-12-03 Mehmet Pinar, Timothy J Horne
Performance-based research funding systems have been extensively used around the globe to allocate funds across higher education institutes (HEIs), which led to an increased amount of literature examining their use. The UK’s Research Excellence Framework (REF) uses a peer-review process to evaluate the research environment, research outputs and non-academic impact of research produced by HEIs to produce
-
Key factors affecting the promotion of researchers of the Argentine Scientific and Technical Research Council (CONICET) Research Evaluation (IF 2.8) Pub Date : 2021-11-30 María Guillermina D'Onofrio, Juan D Rogers
The evaluation system is an important component of the institutional arrangements that may shape the career trajectories of researchers. Using logistic regression and recursive partition models, we analyze the resulting key individual level factors that seem to play an important role in the promotion of researchers through the research evaluation system of the Argentine National Council of Scientific
-
Documenting development of interdisciplinary collaboration among researchers by visualizing connections Research Evaluation (IF 2.8) Pub Date : 2021-11-13 Linden E Higgins, Julia M Smith
In 2015, a multidisciplinary group of academic researchers and extension personnel was awarded a US Department of Agriculture Coordinated Agricultural Project grant. The team’s goal was to understand human decision-making in the context of potential livestock disease outbreaks, investigating the impediments to increased adoption of biosecurity practices by producers from multiple perspectives. Counting
-
Biographical representation, from narrative to list: The evolution of curricula vitae in the humanities, 1950 to 2010 Research Evaluation (IF 2.8) Pub Date : 2021-11-09 Julian Hamann, Wolfgang Kaltenbrunner
Curricula vitae (CVs) are a crucial device for the evaluation of academic personae and biographies. They play a key role in the competitive assessments that underpin the reproduction of the academic workforce. Drawing on 80 CVs which have been part of candidates’ applications for vacant professorships, our article provides a longitudinal study of the development of CVs used by German scholars in professorial
-
Capabilities for transdisciplinary research Research Evaluation (IF 2.8) Pub Date : 2021-11-01 Cian O’Donovan, Aleksandra (Ola) Michalec, Joshua R Moon
Problems framed as societal challenges have provided fresh impetus for transdisciplinary research. In response, funders have started programmes aimed at increasing transdisciplinary research capacity. However, current programme evaluations do not adequately measure the skills and characteristics of individuals and collectives doing this research. Addressing this gap, we propose a systematic framework
-
Measuring societal impact of research—Developing and validating an impact instrument for occupational health and safety Research Evaluation (IF 2.8) Pub Date : 2021-09-30 Ole Henning Sørensen, Jakob Bjørner, Andreas Holtermann, Johnny Dyreborg, Jorid Birkelund Sørli, Jesper Kristiansen, Steffen Bohni Nielsen
Research funders and policymakers increasingly focus on societal benefits of their investments in research. Research institutions thus face increasing pressure to demonstrate their societal impact to prove their legitimacy and worth. To this end, research institutions need reliable, quantitative methods to measure societal impact. This article describes the development and test of an instrument to
-
Evidence of research mastery: How applicants argue the feasibility of their research projects Research Evaluation (IF 2.8) Pub Date : 2021-09-30 Eva Barlösius, Kristina Blem
Although many studies have shown that reviewers particularly value the feasibility of a proposed project, very little attention has gone to how applicants try to establish the plausibility of their proposal’s realization. With a sample of 335 proposals, we examined the ways applicants reason the feasibility of their projects and the kinds of evidence they provide to support those assertions. We identified
-
Do peers share the same criteria for assessing grant applications? Research Evaluation (IF 2.8) Pub Date : 2021-09-23 Sven E Hug, Michael Ochsner
This study examines a basic assumption of peer review, namely, the idea that there is a consensus on evaluation criteria among peers, which is a necessary condition for the reliability of peer judgements. Empirical evidence indicating that there is no consensus or more than one consensus would offer an explanation for the disagreement effect, the low inter-rater reliability consistently observed in
-
Research evaluations for an energy transition? Insights from a review of Swedish research evaluation reports Research Evaluation (IF 2.8) Pub Date : 2021-09-14 Sofie Sandin, Mats Benner
Energy efficiency is identified as a vital area for addressing sustainability challenges of our time. Governments throughout the world invest vast amounts of resources in research, for advancing knowledge on energy efficiency, and for fostering innovations that can support a transition towards a more sustainable energy system. Evaluation can be an important component in transition processes, for setting