-
Transfer versus co-production: Knowledge as ‘MEANS’ to sustainability as an ‘END’ Research Evaluation (IF 2.9) Pub Date : 2024-08-17 Nour Chams, Bouali Guesmi, Mireia Molins i Folch, Rosa Cubel, Jose Maria Gil Roig
This study examines the relationship between sustainability performance and knowledge, as well as between innovation transfer and co-production. It moves away from evaluating processes to exploring impact by investigating how explicit and tacit knowledge archetypes are associated with sustainability performance in the agri-food sector. Applying the knowledge-based theory and the Shannon index, we rely
-
Where do field-specific notions of research quality come from? Research Evaluation (IF 2.9) Pub Date : 2024-08-17 Grit Laudel
Researchers’ notions of research quality depend on their field of research. Previous studies have shown that field-specific assessment criteria exist but could explain neither why these specific criteria and not others exist, nor how criteria are used in specific assessment situations. To give initial answers to these questions, formal assessment situations (reviews of publications) and informal assessment
-
Funding lotteries for research grant allocation: An extended taxonomy and evaluation of their fairness Research Evaluation (IF 2.9) Pub Date : 2024-08-13 Thomas Feliciani, Junwen Luo, Kalpana Shankar
Some research funding organizations (funders) are experimenting with random allocation of funding (funding lotteries), whereby funding is awarded to a random subset of eligible applicants evaluated positively by review panels. There is no consensus on which allocation rule is fairer—traditional peer review or funding lotteries—partly because there exist different ways of implementing funding lotteries
-
Who gets the grant? A persona-based investigation into research funding panelist preferences Research Evaluation (IF 2.9) Pub Date : 2024-08-09 João M Santos
The allocation of scientific funding through grant programs is crucial for research advancement. While independent peer panels typically handle evaluations, their decisions can lean on personal preferences that go beyond the stated criteria, leading to inconsistencies and potential biases. Given these concerns, our study employs a novel method, using simulated, data-driven, and narrative personas of
-
Quality from within: Entry points to research quality in the humanities Research Evaluation (IF 2.9) Pub Date : 2024-08-09 Klara Müller, Linus Salö, Sverker Sörlin
It is well known that research quality notions vary across research fields. Despite this, humanities quality notions are often portrayed as deviant or particularly hard to grasp. To some extent, this has a historical explanation, as notions from within the humanities have not been the standards used in the development of research evaluation tools. Accordingly, we argue that current discussions on research
-
Societal interaction plans—A tool for enhancing societal engagement of strategic research in Finland Research Evaluation (IF 2.9) Pub Date : 2024-06-11 Kirsi Pulkkinen, Timo Aarrevaara, Mikko Rask, Markku Mattila
In this paper we investigate the practices and capacities that define successful societal interaction of research groups with stakeholders in mutually beneficial processes. We studied the Finnish Strategic Research Council’s (SRC) first funded projects through a dynamic governance lens. The aim of the paper is to explore how the societal interaction was designed and commenced at the onset of the projects
-
Research impact assessment: Developing and applying a viable model for the social sciences Research Evaluation (IF 2.9) Pub Date : 2024-05-18 Thomaz Wood, Adriana Wilner
Prompted by research funding agencies and regulatory bodies, universities need to demonstrate and manage the impact of their research. Therefore, models for evaluating research impacts that can be applied quickly and at a reasonable cost are needed. However, the development of these models presents considerable challenges: research impact is multidimensional and can occur over time because of multiple
-
The forced battle between peer-review and scientometric research assessment: Why the CoARA initiative is unsound Research Evaluation (IF 2.9) Pub Date : 2024-05-12 Giovanni Abramo
Endorsed by the European Research Area, a Coalition for Advancing Research Assessment (CoARA), primarily composed of research institutions and funders, was established in 2022. Its mission is to reform research assessment, emphasizing a shift towards qualitative judgment. This study examines the CoARA initiative from theoretical, practical, and applicative perspectives, exploring its implications.
-
Policy variation in the external evaluation of research for tenure at U.S. universities Research Evaluation (IF 2.9) Pub Date : 2024-05-08 Lance Hannon, Meredith Bergey
There is a robust literature documenting differences in peer review processes for scholarly outlets. Knowledge of this variability has provoked thoughtful debate about the best approach for promoting rigor and innovation in scientific research (e.g. single-blind vs. double-blind review, or more recently, double-blind vs. open review). We aim to expand this conversation to external peer review processes
-
Towards higher standardization of funding acknowledgements in scientific publications: Current status in the case of a national and a European research fellowship scheme Research Evaluation (IF 2.9) Pub Date : 2024-04-30 María Bordons, Fernanda Morillo, Belén Álvarez-Bornstein
Acknowledgement of funding sources in scientific publications is becoming mandatory in science. As funders' instructions are often vague, there is great variability in the way authors acknowledge funding, this hindering the linking of grants to their subsequent publications. The aim of this study is to analyse how funding is acknowledged in scientific publications in two important research fellowship
-
The quality landscape of economics: The top five and beyond Research Evaluation (IF 2.9) Pub Date : 2024-04-24 Anders Hylmö, Kody Steffy, Duncan A Thomas, Liv Langfeldt
Whereas a growing number of studies evidence that research quality notions and evaluative practices are field- and context-specific, many focus on single evaluative practices or moments. This paper introduces the concept of quality landscape to capture dynamics of interrelated quality notions, evaluative moments and practices in a research field. This concept shifts focus to (1) the field-specific
-
Do thematic funding instruments lead researchers in new directions? Strategic funding priorities and topic switching among British grant recipients Research Evaluation (IF 2.9) Pub Date : 2024-04-24 Emil Bargmann Madsen, Mathias Wullum Nielsen
Research agencies increasingly use thematic funding schemes to steer scientists toward issues of political interest. Funders set strategic priorities and expect scientists to follow the money. Using statistical-matching methods, we paired 10,475 UK-based recipients of thematic grants (with aims proposed by the funding agency) and responsive-mode grants (with aims proposed by researchers) in Physics
-
Characteristics of contemporary health research practice: A shift from ivory tower to collaborative power Research Evaluation (IF 2.9) Pub Date : 2024-04-09 Daniel P Wadsworth, Alison Craswell, Andy Ward, Kristen Tulloch, Mia A Schaumberg, Lee Stoner, Pamela J Meredith, Christopher D Askew
The academic world is rapidly changing due to multiple influences, including COVID-19, technological advancement, tertiary competition, shifting government policies, and emerging research practices and priorities. Given this rapidly changing tertiary-education landscape, added importance is placed on the need for emerging research leaders to understand contemporary research practices which likely exceeds
-
The impact of researchers’ perceived pressure on their publication strategies Research Evaluation (IF 2.9) Pub Date : 2024-03-25 David Johann, Jörg Neufeld, Kathrin Thomas, Justus Rathmann, Heiko Rauhut
This article investigates researchers’ publication strategies and how their perceived pressure to publish and to obtain external funding are related to these strategies. The analyses rely on data from the Zurich Survey of Academics (ZSoA), an online survey representative of academics working at higher education institutions in Germany, Austria, and Switzerland. The results suggest that academics pursue
-
Practicing responsible research assessment: Qualitative study of faculty hiring, promotion, and tenure assessments in the United States Research Evaluation (IF 2.9) Pub Date : 2024-03-14 Alexander Rushforth, Sarah De Rijcke
Recent times have seen the growth in the number and scope of interacting professional reform movements in science, centered on themes such as open research, research integrity, responsible research assessment, and responsible metrics. The responsible metrics movement identifies the growing influence of quantitative performance indicators as a major problem and seeks to steer and improve practices around
-
Organizational changes and research performance: A multidimensional assessment Research Evaluation (IF 2.9) Pub Date : 2024-02-17 José Luis Jiménez-Andrade, Ricardo Arencibia-Jorge, Miguel Robles-Pérez, Julia Tagüeña, Tzipe Govezensky, Humberto Carrillo-Calvet, Rafael A Barrio, Kimmo Kaski
This paper analyzes the research performance evolution of a scientific institute, from its genesis through various stages of development. The main aim is to obtain, and visually represent, bibliometric evidence of the correlation of organizational changes on the development of its scientific performance; particularly, structural and leadership changes. The study involves six bibliometric indicators
-
What is a high-quality research environment? Evidence from the UK’s research excellence framework Research Evaluation (IF 2.9) Pub Date : 2024-02-16 Matthew Inglis, Elizabeth Gadd, Elizabeth Stokoe
As part of the UK university sector’s performance-related research funding model, the ‘REF’ (Research Excellence Framework), each discipline-derived ‘Unit of Assessment’ must submit a statement to provide information about their environment, culture, and strategy for enabling research and impact. Our aim in this paper is to identify the topics on which these statements focus, and how topic variation
-
Towards a sustainable and responsible model for monitoring open science and research—analysis of the Finnish model for monitoring open science and research Research Evaluation (IF 2.9) Pub Date : 2024-02-16 Laura Himanen, Susanna Nykyri
Advancing and supporting the development of open science practices is of global interest, and subsequently also the monitoring of the ongoing developments is more and more on the agenda. In Finland, monitoring open science has a relatively long history, and the first evaluation of research performing and research funding organisations took place already in 2015. The system has evolved over the years
-
Additional experiments required: A scoping review of recent evidence on key aspects of Open Peer Review Research Evaluation (IF 2.9) Pub Date : 2024-02-09 Tony Ross-Hellauer, Serge P J M Horbach
Diverse efforts are underway to reform the journal peer review system. Combined with growing interest in Open Science practices, Open Peer Review (OPR) has become of central concern to the scholarly community. However, what OPR is understood to encompass and how effective some of its elements are in meeting the expectations of diverse communities, are uncertain. This scoping review updates previous
-
Effective mission-oriented research: A new framework for systemic research impact assessment Research Evaluation (IF 2.9) Pub Date : 2024-02-09 Lena Pfeifer, Katharina Helming
Mission-oriented research combines a wide array of natural and social science disciplines to offer solutions for complex and multi-dimensional challenges such as climate change, loss of biodiversity, and scarcity of natural resources. The utilization of the outputs of mission-oriented research aims for changes in behavior, policy and practice resulting in real world impacts. Systematically assessing
-
The legal foundation of responsible research assessment: An overview on European Union and Italy Research Evaluation (IF 2.9) Pub Date : 2023-11-29 Ginevra Peruginelli, Janne Pölönen
During the past decade, responsible research assessment (RRA) has become a major science policy goal to advance responsible research and innovation and open science. Starting with the DORA declaration in 2012, common understanding of the needs and demands of RRA has been shaped by a growing number of initiatives, culminating in a European Commission supported Agreement on reforming research assessment
-
The conflict of impact for early career researchers planning for a future in the academy Research Evaluation (IF 2.9) Pub Date : 2023-11-22 Marta Natalia Wróblewska, Corina Balaban, Gemma Derrick, Paul Benneworth
It has been argued that due to the growing importance attributed to research impact and forms of its evaluation, an academic ‘culture of impact’ is emerging. It would include certain concepts, values, and skills related to the area of generating and documenting impact. We use thematic and discourse analysis to analyse open answers from 100 questionnaires on research impact submitted by ECRs working
-
Targeted, actionable and fair: Reviewer reports as feedback and its effect on ECR career choices Research Evaluation (IF 2.9) Pub Date : 2023-11-07 Gemma Elizabeth Derrick, Alessandra Zimmermann, Helen Greaves, Jonathan Best, Richard Klavans
Previous studies of the use of peer review for the allocation of competitive funding agencies have concentrated on questions of efficiency and how to make the ‘best’ decision, by ensuring that successful applicants are also the more productive or visible in the long term. This paper examines the components of feedback received from an unsuccessful grant application, is associated with motivating applicants
-
Effects of transdisciplinary research on scientific knowledge and reflexivity Research Evaluation (IF 2.9) Pub Date : 2023-11-06 Oskar Marg, Lena Theiler
Transdisciplinary research (TDR) is conceptualized as not only providing societal effects but also benefiting academia. However, recent literature on the evaluation of TDR has focused almost entirely on the societal effects of TDR. A discussion of the scientific effects of TDR is needed to do justice to the potential of this research mode. To date, little empirical research has focused on the effects
-
Theory of systems change: An initial, middle-range theory of public health research impact Research Evaluation (IF 2.9) Pub Date : 2023-10-19 Melinda Craike, Bojana Klepac, Amy Mowle, Therese Riley
There is increasing attention on evidencing research impact and applying a systems thinking perspective in public health. However, there is limited understanding of the extent to which and how public health research that applies a systems thinking perspective contributes to changes in system behaviour and improved population health outcomes. This paper addresses the theoretical limitations of research
-
The effect of applied research institutes on invention: Evidence from the Fraunhofer centres in Europe Research Evaluation (IF 2.9) Pub Date : 2023-10-13 Pedro Llanos-Paredes
This study examines the impact of the Fraunhofer Society, Europe’s largest network of applied research institutes, on patent applications. A difference-in-differences strategy was employed exploiting the establishment of five new Fraunhofer centres in the 2000s. The panel includes 65,963 European applicants (both firms and independent inventors) between 1980 and 2019. The results show that establishing
-
Warnings of declining research productivity: Does Italy buck the trend? Research Evaluation (IF 2.9) Pub Date : 2023-09-17 Giovanni Abramo, Ciriaco Andrea D’Angelo
The paper takes a scientometric approach to measure the change in research productivity of Italian academics before the COVID-19 pandemic outbreak. We propose a composite output/input bibliometric indicator and apply it at the field level, conducting a longitudinal analysis. Although the number of academics in the national academic system has decreased, we register very strong growth in both the number
-
Do funding modes matter? A multilevel analysis of funding allocation mechanisms on university research performance Research Evaluation (IF 2.9) Pub Date : 2023-09-04 Thomas Zacharewicz, Noemi Pulido Pavón, Luis Antonio Palma Martos, Benedetto Lepori
Over the last decades, most EU countries have profoundly reshaped their public research funding systems by shifting from traditional institutional block-funding towards more project-based mechanisms. The main rationale underlying this evolution builds on the assumption that project funding would foster research performance through the introduction of competitive allocation mechanisms. In contrast with
-
Methods for measuring social and conceptual dimensions of convergence science Research Evaluation (IF 2.9) Pub Date : 2023-07-20 Alexander Michael Petersen, Felber Arroyave, Ioannis Pavlidis
Convergence science is an intrepid form of interdisciplinarity defined by the US National Research Council as ‘the coming together of insights and approaches from originally distinct fields’ to strategically address grand challenges. Despite its increasing relevance to science policy and institutional design, there is still no practical framework for measuring convergence. We address this gap by developing
-
How unpredictable is research impact? Evidence from the UK’s Research Excellence Framework Research Evaluation (IF 2.9) Pub Date : 2023-07-14 Ohid Yaqub, Dmitry Malkov, Josh Siepel
Although ex post evaluation of impact is increasingly common, the extent to which research impacts emerge largely as anticipated by researchers, or as the result of serendipitous and unpredictable processes, is not well understood. In this article, we explore whether predictions of impact made at the funding stage align with realized impact, using data from the UK’s Research Excellence Framework (REF)
-
Turning academics into researchers: The development of National Researcher Categorization Systems in Latin America Research Evaluation (IF 2.9) Pub Date : 2023-07-13 Federico Vasen, Nerina F Sarthou, Silvina A Romano, Brenda D Gutiérrez, Manuel Pintos
Evaluation procedures play a crucial role in science and technology systems, particularly within academic career structures. This article focuses on an approach to evaluation that has gained prominence in Latin America over the past four decades. This scheme assesses the individual performance of academics based on their academic activities and outputs and assigns them a ‘researcher category’, which
-
Can journal reviewers dependably assess rigour, significance, and originality in theoretical papers? Evidence from physics Research Evaluation (IF 2.9) Pub Date : 2023-06-30 Mike Thelwall, Janusz A Hołyst
Peer review is a key gatekeeper for academic journals, attempting to block inadequate submissions or correcting them to a publishable standard, as well as improving those that are already satisfactory. The three key aspects of research quality are rigour, significance, and originality but no prior study has assessed whether journal reviewers are ever able to judge these effectively. In response, this
-
The forms of societal interaction in the social sciences, humanities and arts: Below the tip of the iceberg Research Evaluation (IF 2.9) Pub Date : 2023-06-02 Elea Giménez-Toledo, Julia Olmos-Peñuela, Elena Castro-Martínez, François Perruchas
Science policymakers are devoting increasing attention to enhancing the social valorization of scientific knowledge. Since 2010, several international evaluation initiatives have been implemented to assess knowledge transfer and exchange practices and the societal impacts of research. Analysis of these initiatives would allow investigation of the different knowledge transfer and exchange channels and
-
Boundary-work and social closure in academic recruitment: Insights from the transdisciplinary subject area Swedish as a Second Language Research Evaluation (IF 2.9) Pub Date : 2023-05-15 Natalia Ganuza, Linus Salö
This article explores practices of evaluation in academic recruitment in Swedish as a Second Language (SSL), an expanding and transdisciplinary subject area. As is common elsewhere, Swedish academia relies on a tradition of external expert review intended to ensure a meritocratic process. Here, we present an analysis of 109 written expert reports concerning recruitment to 57 positions in SSL during
-
The missing links of research impact Research Evaluation (IF 2.9) Pub Date : 2023-05-12 David Budtz Pedersen, Rolf Hvidtfeldt
In this article, we present a conceptual framework for studying research impact focusing on the foundations that need to be in place to accelerate an observable change of policy, practice or behaviour. The article investigates the relationship between micro-impacts and societal change, and how smaller impacts scale into larger cascades of end effects and value creation. We define micro-impacts as interactions
-
Promoting narrative CVs to improve research evaluation? A review of opinion pieces and experiments Research Evaluation (IF 2.9) Pub Date : 2023-04-19 Frédérique Bordignon, Lauranne Chaignon, Daniel Egret
As the academic community has become increasingly concerned about the drifts of research evaluation, mostly researchers’ evaluation, because of the overreliance on metrics, many expert groups have made recommendations to improve the way researchers should be evaluated. In this study, we focus on the recommendation to use narrative curriculum vitae (CVs). We review 28 opinion pieces and 7 experiments
-
The price of quality: Scholarly publishing business is the primary predictor of citation-based indicators of journal performance in ecology and evolutionary biology Research Evaluation (IF 2.9) Pub Date : 2023-04-19 Adriana Ruggiero
Citation-based indicators of journals’ performance are often assumed to offer an objective, albeit indirect, way of measuring research quality. However, recent concerns about their applicability for research evaluation suggested these indicators could depend on historical and socioeconomic factors associated with scholarly publishing tradition and business, respectively. The present study addressed
-
Improving universities’ activities in academic startup support through public interventions: The effectiveness of the German programme ‘EXIST—leverage of potentials’ Research Evaluation (IF 2.9) Pub Date : 2023-03-29 Christoph E Mueller
Academic startups have a considerable economic impact, which is why public support programmes for them are considered an important component of innovation and technology policy. In this context, university support programmes can be an important part of the policy toolkit by aiming to improve universities’ startup environment and thus promote startup activity at those institutions. Assessing the effectiveness
-
Changing conceptualization of innovation in the European Union and its impact on universities: Critical junctures and evolving institutional demands Research Evaluation (IF 2.9) Pub Date : 2023-03-17 Ridvan Cinar, Paul Benneworth, Lars Coenen
This article explores underlying mechanisms triggering a change in conceptualization of innovation in the European Union (EU), the impact of this change on institutional demands upon European universities and implications for evaluation procedures. We mobilize the theoretical concept of critical junctures to explore significant periods that have affected understanding of innovation in the EU as well
-
Gender diversity and publication activity—an analysis of STEM in the UK Research Evaluation (IF 2.9) Pub Date : 2023-03-09 Yasaman Sarabi, Matthew Smith
Gender diversity in STEM remains a significant issue, as the field continues to be a male dominated one, despite increased attention on the subject. This article examines the interplay between gender diversity on projects funded by a major UK research council, the Engineering and Physical Sciences Research Council, and the publication activity of a project, as measured by the average journal quality
-
Early career academic's odyssey: A narrative study of her professional identity construction Research Evaluation (IF 2.9) Pub Date : 2023-03-08 Javier Mula-Falcón, Katia Caballero
Recent decades have seen a major transformation of the Spanish university system caused by changes introduced in the teaching staff evaluation procedure in which research has been prioritized. As a result, there has been a growing interest in studying how these procedures impact the way in which early career academics construct their professional identity. In this vein, this article aims to analyze
-
Assessing the variety of collaborative practices in translational research: An analysis of scientists’ ego-networks Research Evaluation (IF 2.9) Pub Date : 2023-03-08 Adrián A Díaz-Faes, Oscar Llopis, Pablo D’Este, Jordi Molas-Gallart
Translational research policies aim to reshape how biomedical scientists organize, conceive, and conduct science in order to accelerate healthcare improvements and medical innovations. Yet most analyses and evaluations of these initiatives focus on measuring the outputs generated in the different stages of the research process rather than observing scientists’ research practices directly. In this article
-
On the societal impact of publicly funded Circular Bioeconomy research in Europe Research Evaluation (IF 2.9) Pub Date : 2023-02-23 Ana Sofia Brandão, José M R C A Santos
Europe has taken a world leadership position in setting policy priorities for Circular Bioeconomy (CBE) as a key determinant of economic, social, and environmental sustainability. Consequently, European R&D investment in this area keeps growing along with the societal pressure to demonstrate the return of investment of publicly funded projects. Thus, this work presents a pioneering exploratory analysis
-
The rocky road to translational science: An analysis of Clinical and Translational Science Awards Research Evaluation (IF 2.9) Pub Date : 2023-01-19 Yeon Hak Kim, You-Na Lee, Seokkyun Woo
Studies point out that the productivity decline in biomedicine is in significant part due to difficulties in translating basic science into clinical application. To promote translational research, the US NIH launched the Clinical and Translational Science Awards (CTSA) program in 2006. Prior evaluations of the CTSA program often assumed that the key to translation is inter-organizational collaboration
-
Describing the state of a research network: A mixed methods approach to network evaluation. Research Evaluation (IF 2.9) Pub Date : 2022-10-28 James M Bowen,Mathieu Ouimet,Justin Lawarée,Joanna Bielecki,Ashley Rhéaume,Caylee Greenberg,Valeria E Rac
Diabetes Action Canada Strategy for Patient-Oriented Research (SPOR) Network in Chronic Disease was formed in 2016 and is funded primarily through the Canadian Institutes of Health Research (CIHR). We propose a novel mixed-methods approach to a network evaluation integrating the State of Network Evaluation framework and the Canadian Academy of Health Sciences (CAHS) preferred framework and indicators
-
Explaining employment sector choices of doctoral graduates in Germany Research Evaluation (IF 2.9) Pub Date : 2022-10-03 Lea Goldan, Steffen Jaksztat, Christiane Gross
Previous research in different national contexts has shown that individual preferences for certain job attributes, objective labour market conditions, subjective career prospects, and external encouragement shape doctoral graduates’ career decisions. For Germany, where the number of awarded doctoral degrees is highest within the European Union and where no established academic tenure-track system exists
-
Research governance and the dynamics of science: A framework for the study of governance effects on research fields Research Evaluation (IF 2.9) Pub Date : 2022-08-19 Maria Nedeva, Mayra M Tirado, Duncan A Thomas
This article offers a framework for the study of research governance effects on scientific fields framed by notions of research quality and the epistemic, organizational, and career choices they entail. The framework interprets the contested idea of ‘quality’ as an interplay involving notion origins, quality attributes, and contextual sites. We mobilize the origin and site components, to frame organizational-level
-
The footprint of a metrics-based research evaluation system on Spain’s philosophical scholarship: An analysis of researchers’ perceptions Research Evaluation (IF 2.9) Pub Date : 2022-08-12 Ramón A Feenstra, Emilio Delgado López-Cózar
The use of bibliometric indicators in research evaluation has a series of complex impacts on academic inquiry. These systems have gradually spread into a wide range of locations and disciplines, including the humanities. The aim of this study is to examine their effects as perceived by philosophy and ethics researchers in Spain, a country where bibliometric indicators have long been used to evaluate
-
The impact of Italian performance-based research funding systems on the intensity of international research collaboration Research Evaluation (IF 2.9) Pub Date : 2022-07-29 Giovanni Abramo, Ciriaco Andrea D’Angelo
The study of national research assessment exercises serves to evaluate the effectiveness of policies versus their objectives and to improve the formulation of future initiatives. The aim of the current study is to verify whether the introduction of the first performance-based research funding in Italy, based on the 2004–10 VQR assessment, achieved the intended objective of inducing greater international
-
Quis judicabit ipsos judices? A case study on the dynamics of competitive funding panel evaluations Research Evaluation (IF 2.9) Pub Date : 2022-07-18 João M Santos
Securing research funding is essential for all researchers. The standard evaluation method for competitive grants is through evaluation by a panel of experts. However, the literature notes that peer review has inherent flaws and is subject to biases, which can arise from differing interpretations of the criteria, the impossibility for a group of reviewers to be experts in all possible topics within
-
Evaluation of the arts in performance-based research funding systems: An international perspective Research Evaluation (IF 2.9) Pub Date : 2022-07-18 Kamila Lewandowska, Emanuel Kulczycki, Michael Ochsner
This article provides a comprehensive analysis of the evaluation of the arts within performance-based research funding systems (PRFSs). Previous literature on PRFSs has overlooked the arts and focussed primarily on outputs in relation to the sciences and humanities. We develop a typology of how artistic outputs are evaluated within 10 countries’ PRFSs, operating in Australia, the Czech Republic, Italy
-
Valorization of transdisciplinary research: An evaluation approach and empirical illustration Research Evaluation (IF 2.9) Pub Date : 2022-06-30 Stefania Munaretto, Caro E Mooren, Laurens K Hessels
In recent times, there has been a surge of impact-oriented, transdisciplinary research programmes and projects integrating multiple disciplines, types of knowledge and practices. An essential element often mentioned in the literature to improve the performance of these programmes and support impact delivery is continuous reflection and learning via evaluation. We argue that because a standard format
-
Under pressure: The extent and distribution of perceived pressure among scientists in Germany, Austria, and Switzerland Research Evaluation (IF 2.9) Pub Date : 2022-06-21 David Johann, Isabel J Raabe, Heiko Rauhut
While it has been stressed repeatedly that academics nowadays have come to face extensive pressure, the extent and distribution of pressure to publish and to secure third-party funding has not been systematically investigated on a large scale. Based on the Zurich Survey of Academics, a representative large-scale web survey among academics working at universities in Germany, Austria, and Switzerland
-
University policy engagement bodies in the UK and the variable meanings of and approaches to impact Research Evaluation (IF 2.9) Pub Date : 2022-06-16 Hannah Durrant, Eleanor MacKillop
Over the last decade, there has been a proliferation of policy engagement bodies set up within universities worldwide. The present study focuses on the British experience of this phenomenon but with relevance to other contexts. Multiple factors are at play to explain this growth, from the Research Excellence Framework impact agenda (which assesses and ranks the quality of research in UK universities
-
Automated citation recommendation tools encourage questionable citations Research Evaluation (IF 2.9) Pub Date : 2022-06-02 Serge P J M Horbach, Freek J W Oude Maatman, Willem Halffman, Wytske M Hepkema
Citing practices have long been at the heart of scientific reporting, playing both socially and epistemically important functions in science. While such practices have been relatively stable over time, recent attempts to develop automated citation recommendation tools have the potential to drastically impact citing practices. We claim that, even though such tools may come with tempting advantages,
-
A participatory approach to tracking system transformation in clusters and innovation ecosystems—Evolving practice in Sweden’s Vinnväxt programme Research Evaluation (IF 2.9) Pub Date : 2022-04-01 Emily Wise,Moa Eklund,Madeline Smith,James Wilson
Abstract For decades, cluster initiatives and funding programmes have been used as instruments of industrial and innovation policy—addressing system failures by strengthening linkages among actors, fostering innovation, and developing more effective innovation systems. More recently, a growing segment of these initiatives are also focused on driving system-level transformation and contributing to broader
-
Interdisciplinary research and policy impacts: Assessing the significance of knowledge coproduction Research Evaluation (IF 2.9) Pub Date : 2022-03-25 Luisa Veras de Sandes-Guimarães, Raquel Velho, Guilherme Ary Plonski
Research impact on various societal spheres has been increasingly demanded for funding purposes and as a form of demonstrating the relevance of scientific research for societal problems. In this context, interest in interdisciplinary research and knowledge coproduction has grown as a path to achieve this expected impact, but few studies have considered its conjoint association with societal impact
-
Research calls, competition for funding and inefficiency Research Evaluation (IF 2.9) Pub Date : 2022-03-24 António Osório,Lutz Bornmann
Abstract Research groups spend time and resources in the process of applying for funding. This issue raises important questions regarding inefficiency and whether the currently used funding mechanisms are adequate. This article aims to identify ways of reducing the inefficiency and the waste of resources when making research funding calls. We look at four ways of reducing inefficiency. Inefficiency
-
Predicting funded research project performance based on machine learning Research Evaluation (IF 2.9) Pub Date : 2022-03-15 Hoon Jang
Abstract Increasing investment and interest in research and development (R&D) requires an efficient management system for achieving better research project outputs. In tandem with this trend, there is a growing need to develop a method for predicting research project outputs. Motivated by this, using information gathered in the early stage of projects, this study addresses the problem of predicting
-
Does R&D tax credit impact firm behaviour? Micro evidence for Portugal Research Evaluation (IF 2.9) Pub Date : 2022-01-27 Alexandre Paredes, Joana Mendonça, Fernando Bação, Bruno Damásio
In this study, we use panel data to analyse the impact of an R&D tax credit on R&D personnel, particularly the impact on Ph.D. holders allocation, comparing low R&D intensity firms with medium-high and high R&D intensity firms. The results show that, in medium-high and high R&D intensity firms, the R&D tax credit had a significant impact on allocating Ph.D. holders in firms after 3 years of participation