-
Development of a search filter to retrieve reports of interrupted time series studies from MEDLINE and PubMed Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-17 Phi-Yen Nguyen, Joanne E. McKenzie, Simon L. Turner, Matthew J. Page, Steve McDonald
Interrupted time series (ITS) studies contribute importantly to systematic reviews of population-level interventions. We aimed to develop and validate search filters to retrieve ITS studies in MEDLINE and PubMed.
-
Can large language models replace humans in systematic reviews? Evaluating GPT‐4's efficacy in screening and extracting data from peer‐reviewed and grey literature in multiple languages Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-14 Qusai Khraisha, Sophie Put, Johanna Kappenberg, Azza Warraitch, Kristin Hadfield
Systematic reviews are vital for guiding practice, research and policy, although they are often slow and labour‐intensive. Large language models (LLMs) could speed up and automate systematic reviews, but their performance in such tasks has yet to be comprehensively evaluated against humans, and no study has tested Generative Pre‐Trained Transformer (GPT)‐4, the biggest LLM so far. This pre‐registered
-
Easy‐Amanida: An R Shiny application for the meta‐analysis of aggregate results in clinical metabolomics using Amanida and Webchem Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-14 Maria Llambrich, Pau Satorra, Eudald Correig, Josep Gumà, Jesús Brezmes, Cristian Tebé, Raquel Cumeras
Meta‐analysis is a useful tool in clinical research, as it combines the results of multiple clinical studies to improve precision when answering a particular scientific question. While there has been a substantial increase in publications using meta‐analysis in various clinical research topics, the number of published meta‐analyses in metabolomics is significantly lower compared to other omics disciplines
-
LFK index does not reliably detect small‐study effects in meta‐analysis: A simulation study Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-11 Guido Schwarzer, Gerta Rücker, Cristina Semaca
The LFK index has been promoted as an improved method to detect bias in meta‐analysis. Putatively, its performance does not depend on the number of studies in the meta‐analysis. We conducted a simulation study, comparing the LFK index test to three standard tests for funnel plot asymmetry in settings with smaller or larger group sample sizes. In general, false positive rates of the LFK index test markedly
-
-
Correction to “Network Meta-Interpolation: Effect modification adjustment in network meta-analysis using subgroup analyses” Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-03
Ofir Harari, Mohsen Soltanifar, Joseph C. Cappelleri, Andre Verhoek, Mario Ouwens, Caitlin Daly, and Bart Heeg (2023) Network Meta-Interpolation: Effect modification adjustment in network meta-analysis using subgroup analyses. Research Synthesis Methods, 14: 211–233. On page 215, Equation (13) for the best linear unbiased predictor (BLUP) was mistakenly written as xj*(xi)=ρ̂x¯1,x¯2σ̂x¯1σ̂x¯2(xi−x¯j)+x¯j
-
Data extraction for evidence synthesis using a large language model: A proof‐of‐concept study Res. Synth. Methods (IF 9.8) Pub Date : 2024-03-03 Gerald Gartlehner, Leila Kahwati, Rainer Hilscher, Ian Thomas, Shannon Kugley, Karen Crotty, Meera Viswanathan, Barbara Nussbaumer‐Streit, Graham Booth, Nathaniel Erskine, Amanda Konet, Robert Chew
Data extraction is a crucial, yet labor‐intensive and error‐prone part of evidence synthesis. To date, efforts to harness machine learning for enhancing efficiency of the data extraction process have fallen short of achieving sufficient accuracy and usability. With the release of large language models (LLMs), new possibilities have emerged to increase efficiency and accuracy of data extraction for
-
Estimating the extent of selective reporting: An application to economics Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-21 Stephan B. Bruns, Teshome K. Deressa, T. D. Stanley, Chris Doucouliagos, John P. A. Ioannidis
Using a sample of 70,399 published p‐values from 192 meta‐analyses, we empirically estimate the counterfactual distribution of p‐values in the absence of any biases. Comparing observed p‐values with counterfactually expected p‐values allows us to estimate how many p‐values are published as being statistically significant when they should have been published as non‐significant. We estimate the extent
-
A comprehensive review and shiny application on the matching‐adjusted indirect comparison Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-21 Ziren Jiang, Joseph C. Cappelleri, Margaret Gamalo, Yong Chen, Neal Thomas, Haitao Chu
Population‐adjusted indirect comparison (PAIC) is an increasingly used technique for estimating the comparative effectiveness of different treatments for the health technology assessments when head‐to‐head trials are unavailable. Three commonly used PAIC methods include matching‐adjusted indirect comparison (MAIC), simulated treatment comparison (STC), and multilevel network meta‐regression (ML‐NMR)
-
Impact of trial attrition rates on treatment effect estimates in chronic inflammatory diseases: A meta-epidemiological study Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-13 Silja H. Overgaard, Caroline M. Moos, John P. A. Ioannidis, George Luta, Johannes I. Berg, Sabrina M. Nielsen, Vibeke Andersen, Robin Christensen
The objective of this meta-epidemiological study was to explore the impact of attrition rates on treatment effect estimates in randomised trials of chronic inflammatory diseases (CID) treated with biological and targeted synthetic disease-modifying drugs. We sampled trials from Cochrane reviews. Attrition rates and primary endpoint results were retrieved from trial publications; Odds ratios (ORs) were
-
Meta-analyses of partial correlations are biased: Detection and solutions Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-11 T. D. Stanley, Hristos Doucouliagos, Tomas Havranek
We demonstrate that all meta-analyses of partial correlations are biased, and yet hundreds of meta-analyses of partial correlation coefficients (PCCs) are conducted each year widely across economics, business, education, psychology, and medical research. To address these biases, we offer a new weighted average, UWLS+3. UWLS+3 is the unrestricted weighted least squares weighted average that makes an
-
Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-07 František Bartoš, Maximilian Maier, Eric-Jan Wagenmakers, Franziska Nippold, Hristos Doucouliagos, John P. A. Ioannidis, Willem M. Otte, Martina Sladekova, Teshome K. Deresssa, Stephan B. Bruns, Daniele Fanelli, T. D. Stanley
Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated
-
Bayesian meta-analysis for evaluating treatment effectiveness in biomarker subgroups using trials of mixed patient populations Res. Synth. Methods (IF 9.8) Pub Date : 2024-02-05 Lorna Wheaton, Dan Jackson, Sylwia Bujkiewicz
During drug development, evidence can emerge to suggest a treatment is more effective in a specific patient subgroup. Whilst early trials may be conducted in biomarker-mixed populations, later trials are more likely to enroll biomarker-positive patients alone, thus leading to trials of the same treatment investigated in different populations. When conducting a meta-analysis, a conservative approach
-
P-hacking in meta-analyses: A formalization and new meta-analytic methods Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-25 Maya B. Mathur
As traditionally conceived, publication bias arises from selection operating on a collection of individually unbiased estimates. A canonical form of such selection across studies (SAS) is the preferential publication of affirmative studies (i.e., those with significant, positive estimates) versus nonaffirmative studies (i.e., those with nonsignificant or negative estimates). However, meta-analyses
-
Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-23 Andrija Babić, Ognjen Barcot, Tomislav Visković, Frano Šarić, Aleksandar Kirkovski, Ivana Barun, Zvonimir Križanac, Roshan Arjun Ananda, Yuli Viviana Fuentes Barreiro, Narges Malih, Daiana Anne-Marie Dimcea, Josipa Ordulj, Ishanka Weerasekara, Matteo Spezia, Marija Franka Žuljević, Jelena Šuto, Luca Tancredi, Anđela Pijuk, Susanna Sammali, Veronica Iascone, Thilo von Groote, Tina Poklepović Peričić
Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the
-
Twenty years of network meta-analysis: Continuing controversies and recent developments Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-18 A. E. Ades, Nicky J. Welton, Sofia Dias, David M. Phillippo, Deborah M. Caldwell
Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily
-
Appropriateness of conducting and reporting random-effects meta-analysis in oncology Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-14 Jinma Ren, Jia Ma, Joseph C. Cappelleri
A random-effects model is often applied in meta-analysis when considerable heterogeneity among studies is observed due to the differences in patient characteristics, timeframe, treatment regimens, and other study characteristics. Since 2014, the journals Research Synthesis Methods and the Annals of Internal Medicine have published a few noteworthy papers that explained why the most widely used method
-
Using qualitative comparative analysis as a mixed methods synthesis in systematic mixed studies reviews: Guidance and a worked example Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-09 Reem El Sherif, Pierre Pluye, Quan Nha Hong, Benoit Rihoux
Qualitative comparative analysis (QCA) is a hybrid method designed to bridge the gap between qualitative and quantitative research in a case-sensitive approach that considers each case holistically as a complex configuration of conditions and outcomes. QCA allows for multiple conjunctural causation, implying that it is often a combination of conditions that produces an outcome, that multiple pathways
-
Enhancing recall in automated record screening: A resampling algorithm Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-07 Zhipeng Hou, Elizabeth Tipton
Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with
-
Advancing the methodology of mapping reviews: A scoping review Res. Synth. Methods (IF 9.8) Pub Date : 2024-01-02 Hanan Khalil, Fiona Campbell, Katrina Danial, Danielle Pollock, Zachary Munn, Vivian Welsh, Ashrita Saran, Dimi Hoppe, Andrea C. Tricco
This scoping review aims to identify and systematically review published mapping reviews to assess their commonality and heterogeneity and determine whether additional efforts should be made to standardise methodology and reporting. The following databases were searched; Ovid MEDLINE, Embase, CINAHL, PsycINFO, Campbell collaboration database, Social Science Abstracts, Library and Information Science
-
Meta-analysis and partial correlation coefficients: A matter of weights Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-29 Sanghyun Hong, W. Robert Reed
This study builds on the simulation framework of a recent paper by Stanley and Doucouliagos (Research Synthesis Methods 2023;14;515–519). S&D use simulations to make the argument that meta-analyses using partial correlation coefficients (PCCs) should employ a “suboptimal” estimator of the PCC standard error when constructing weights for fixed effect and random effects estimation. We address concerns
-
Bayesian random-effects meta-analysis with empirical heterogeneity priors for application in health technology assessment with very few studies Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-28 Jona Lilienthal, Sibylle Sturtz, Christoph Schürmann, Matthias Maiworm, Christian Röver, Tim Friede, Ralf Bender
In Bayesian random-effects meta-analysis, the use of weakly informative prior distributions is of particular benefit in cases where only a few studies are included, a situation often encountered in health technology assessment (HTA). Suggestions for empirical prior distributions are available in the literature but it is unknown whether these are adequate in the context of HTA. Therefore, a database
-
Consensus on the definition and assessment of external validity of randomized controlled trials: A Delphi study Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-25 Andres Jung, Tobias Braun, Susan Armijo-Olivo, Dimitris Challoumas, Kerstin Luedtke
External validity is an important parameter that needs to be considered for decision making in health research, but no widely accepted measurement tool for the assessment of external validity of randomized controlled trials (RCTs) exists. One of the most limiting factors for creating such a tool is probably the substantial heterogeneity and lack of consensus in this field. The objective of this study
-
Assessment of temporal instability in the applied ecology and conservation evidence base Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-19 Elizabeth Brisco, Elena Kulinskaya, Julia Koricheva
Outcomes of meta-analyses are increasingly used to inform evidence-based decision making in various research fields. However, a number of recent studies have reported rapid temporal changes in magnitude and significance of the reported effects which could make policy-relevant recommendations from meta-analyses to quickly go out of date. We assessed the extent and patterns of temporal trends in magnitude
-
Automated data analysis of unstructured grey literature in health research: A mapping review Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-19 Lena Schmidt, Saleh Mohamed, Nick Meader, Jaume Bacardit, Dawn Craig
The amount of grey literature and ‘softer’ intelligence from social media or websites is vast. Given the long lead-times of producing high-quality peer-reviewed health information, this is causing a demand for new ways to provide prompt input for secondary research. To our knowledge, this is the first review of automated data extraction methods or tools for health-related grey literature and soft data
-
How trace plots help interpret meta-analysis results Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-15 Christian Röver, David Rindskopf, Tim Friede
The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article, we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of τ $$ \tau $$ , the between-study standard deviation, and the shrunken estimates of the study effects as a function of τ $$ \tau $$ . With a small or moderate
-
A study of search strategy availability statements and sharing practices for systematic reviews: Ask and you might receive Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-14 Christine J. Neilson, Zahra Premji
The literature search underpins data collection for all systematic reviews (SRs). The SR reporting guideline PRISMA, and its extensions, aim to facilitate research transparency and reproducibility, and ultimately improve the quality of research, by instructing authors to provide specific research materials and data upon publication of the manuscript. Search strategies are one item of data that are
-
Four alternative methodologies for simulated treatment comparison: How could the use of simulation be re-invigorated? Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-17 Landan Zhang, Sylwia Bujkiewicz, Dan Jackson
Simulated treatment comparison (STC) is an established method for performing population adjustment for the indirect comparison of two treatments, where individual patient data (IPD) are available for one trial but only aggregate level information is available for the other. The most commonly used method is what we call ‘standard STC’. Here we fit an outcome model using data from the trial with IPD
-
metamedian: An R package for meta-analyzing studies reporting medians Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-10 Sean McGrath, XiaoFei Zhao, Omer Ozturk, Stephan Katzenschlager, Russell Steele, Andrea Benedetti
When performing an aggregate data meta-analysis of a continuous outcome, researchers often come across primary studies that report the sample median of the outcome. However, standard meta-analytic methods typically cannot be directly applied in this setting. In recent years, there has been substantial development in statistical methods to incorporate primary studies reporting sample medians in meta-analysis
-
Methods for using Bing's AI-powered search engine for data extraction for a systematic review Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-08 James Edward Hill, Catherine Harris, Andrew Clegg
Data extraction is a time-consuming and resource-intensive task in the systematic review process. Natural language processing (NLP) artificial intelligence (AI) techniques have the potential to automate data extraction saving time and resources, accelerating the review process, and enhancing the quality and reliability of extracted data. In this paper, we propose a method for using Bing AI and Microsoft
-
Network meta analysis to predict the efficacy of an approved treatment in a new indication Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-03 Jennifer L. Proper, Haitao Chu, Purvi Prajapati, Michael D. Sonksen, Thomas A. Murray
Drug repurposing refers to the process of discovering new therapeutic uses for existing medicines. Compared to traditional drug discovery, drug repurposing is attractive for its speed, cost, and reduced risk of failure. However, existing approaches for drug repurposing involve complex, computationally-intensive analytical methods that are not widely used in practice. Instead, repurposing decisions
-
Predatory journals and their practices present a conundrum for systematic reviewers and evidence synthesisers of health research: A qualitative descriptive study Res. Synth. Methods (IF 9.8) Pub Date : 2023-12-03 Danielle Pollock, Timothy Hugh Barker, Jennifer C Stone, Edoardo Aromataris, Miloslav Klugar, Anna M Scott, Cindy Stern, Amanda Ross-White, Ashley Whitehorn, Rick Wiechula, Larissa Shamseer, Zachary Munn
Predatory journals are a blemish on scholarly publishing and academia and the studies published within them are more likely to contain data that is false. The inclusion of studies from predatory journals in evidence syntheses is potentially problematic due to this propensity for false data to be included. To date, there has been little exploration of the opinions and experiences of evidence synthesisers
-
A REML method for the evidence-splitting model in network meta-analysis Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-30 Hans-Peter Piepho, Johannes Forkman, Waqas Ahmed Malik
Checking for possible inconsistency between direct and indirect evidence is an important task in network meta-analysis. Recently, an evidence-splitting (ES) model has been proposed, that allows separating direct and indirect evidence in a network and hence assessing inconsistency. A salient feature of this model is that the variance for heterogeneity appears in both the mean and the variance structure
-
Adapting how to use Google Search to identify studies for systematic reviews in view of a recent change to how search results are displayed. Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-29 Simon Briscoe,Rebecca Abbott,Hassanat Lawal,Morwenna Rogers,Liz Shaw,Jo Thompson Coon
-
Appraisal methods and outcomes of AMSTAR 2 assessments in overviews of systematic reviews of interventions in the cardiovascular field: A methodological study Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-13 Paschalis Karakasis, Konstantinos I. Bougioukas, Konstantinos Pamporis, Nikolaos Fragakis, Anna-Bettina Haidich
This study aimed to assess the methods and outcomes of The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 appraisals in overviews of reviews (overviews) of interventions in the cardiovascular field and identify factors that are associated with these outcomes. MEDLINE, Scopus, and the Cochrane Database of Systematic Reviews were searched until November 2022. Eligible were overviews of cardiovascular
-
Generalizing some key results from “alternative weighting schemes when performing matching-adjusted indirect comparisons” Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-13 Landan Zhang, Dan Jackson
A recent paper proposed an alternative weighting scheme when performing matching-adjusted indirect comparisons. This alternative approach follows the conventional one in matching the covariate means across two studies but differs in that it maximizes the effective sample size when doing so. The appendix of this paper showed, assuming there is one covariate and negative weights are permitted, that the
-
The impact of correction methods on rare-event meta-analysis Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-09 Brinley N. Zabriskie, Nolan Cole, Jacob Baldauf, Craig Decker
Meta-analyses have become the gold standard for synthesizing evidence from multiple clinical trials, and they are especially useful when outcomes are rare or adverse since individual trials often lack sufficient power to detect a treatment effect. However, when zero events are observed in one or both treatment arms in a trial, commonly used meta-analysis methods can perform poorly. Continuity corrections
-
Avoiding common mistakes in meta-analysis: Understanding the distinct roles of Q, I-squared, tau-squared, and the prediction interval in reporting heterogeneity Res. Synth. Methods (IF 9.8) Pub Date : 2023-11-08 Michael Borenstein
In any meta-analysis, it is critically important to report the dispersion in effects as well as the mean effect. If an intervention has a moderate clinical impact on average we also need to know if the impact is moderate for all relevant populations, or if it varies from trivial in some to major in others. Or indeed, if the intervention is beneficial in some cases but harmful in others. Researchers
-
A framework to characterise the reproducibility of meta-analysis results with its application to direct oral anticoagulants in the acute treatment of venous thromboembolism Res. Synth. Methods (IF 9.8) Pub Date : 2023-10-17 Céline Chapelle, Gwénaël Le Teuff, Paul Jacques Zufferey, Silvy Laporte, Edouard Ollier
The number of meta-analyses of aggregate data has dramatically increased due to the facility of obtaining data from publications and the development of free, easy-to-use, and specialised statistical software. Even when meta-analyses include the same studies, their results may vary owing to different methodological choices. Assessment of the replication of meta-analysis provides an example of the variation
-
Use of multiple covariates in assessing treatment-effect modifiers: A methodological review of individual participant data meta-analyses Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-28 Peter J. Godolphin, Nadine Marlin, Chantelle Cornett, David J. Fisher, Jayne F. Tierney, Ian R. White, Ewelina Rogozińska
Individual participant data (IPD) meta-analyses of randomised trials are considered a reliable way to assess participant-level treatment effect modifiers but may not make the best use of the available data. Traditionally, effect modifiers are explored one covariate at a time, which gives rise to the possibility that evidence of treatment-covariate interaction may be due to confounding from a different
-
Catchii: Empowering literature review screening in healthcare Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-28 Andreas Halman, Alicia Oshlack
A systematic review is a type of literature review that aims to collect and analyse all available evidence from the literature on a particular topic. The process of screening and identifying eligible articles from the vast amounts of literature is a time-consuming task. Specialised software has been developed to aid in the screening process and save significant time and labour. However, the most suitable
-
Accuracy and precision of fixed and random effects in meta-analyses of randomized control trials for continuous outcomes Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-26 Timo Gnambs, Ulrich Schroeders
Meta-analyses of treatment effects in randomized control trials are often faced with the problem of missing information required to calculate effect sizes and their sampling variances. Particularly, correlations between pre- and posttest scores are frequently not available. As an ad-hoc solution, researchers impute a constant value for the missing correlation. As an alternative, we propose adopting
-
A comparison of machine learning methods to find clinical trials for inclusion in new systematic reviews from their PROSPERO registrations prior to searching and screening Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-25 Shifeng Liu, Florence T. Bourgeois, Claire Narang, Adam G. Dunn
Searching for trials is a key task in systematic reviews and a focus of automation. Previous approaches required knowing examples of relevant trials in advance, and most methods are focused on published trial articles. To complement existing tools, we compared methods for finding relevant trial registrations given a International Prospective Register of Systematic Reviews (PROSPERO) entry and where
-
Sensitivity analysis for the interactive effects of internal bias and publication bias in meta-analyses Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-24 Maya B. Mathur
Meta-analyses can be compromised by studies' internal biases (e.g., confounding in nonrandomized studies) as well as publication bias. These biases often operate nonadditively: publication bias that favors significant, positive results selects indirectly for studies with more internal bias. We propose sensitivity analyses that address two questions: (1) “For a given severity of internal bias across
-
Evaluation of statistical methods used to meta-analyse results from interrupted time series studies: A simulation study Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-20 Elizabeth Korevaar, Simon L. Turner, Andrew B. Forbes, Amalia Karahalios, Monica Taljaard, Joanne E. McKenzie
Interrupted time series (ITS) are often meta-analysed to inform public health and policy decisions but examination of the statistical methods for ITS analysis and meta-analysis in this context is limited. We simulated meta-analyses of ITS studies with continuous outcome data, analysed the studies using segmented linear regression with two estimation methods [ordinary least squares (OLS) and restricted
-
Robust variance estimation in small meta-analysis with the standardized mean difference Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-17 Rrita Zejnullahi, Larry V. Hedges
Conventional random-effects models in meta-analysis rely on large sample approximations instead of exact small sample results. While random-effects methods produce efficient estimates and confidence intervals for the summary effect have correct coverage when the number of studies is sufficiently large, we demonstrate that conventional methods result in confidence intervals that are not wide enough
-
How to plan and manage an individual participant data meta-analysis. An illustrative toolkit Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-12 Lauren Maxwell, Priya Shreedhar, Mabel Carabali, Brooke Levis
Individual participant data meta-analyses (IPD-MAs) have several benefits over standard aggregate data meta-analyses, including the consideration of additional participants, follow-up time, and the joint consideration of study- and participant-level heterogeneity for improved diagnostic and prognostic model development and evaluation. However, IPD-MAs are resource-intensive and require careful budgeting
-
What are the best methods for rapid reviews of the research evidence? A systematic review of reviews and primary studies Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-11 Michelle M. Haby, Jorge Otávio Maia Barreto, Jenny Yeon Hee Kim, Sasha Peiris, Cristián Mansilla, Marcela Torres, Diego Emmanuel Guerrero-Magaña, Ludovic Reveiz
Rapid review methodology aims to facilitate faster conduct of systematic reviews to meet the needs of the decision-maker, while also maintaining quality and credibility. This systematic review aimed to determine the impact of different methodological shortcuts for undertaking rapid reviews on the risk of bias (RoB) of the results of the review. Review stages for which reviews and primary studies were
-
Causally interpretable meta-analysis: Clearly defined causal effects and two case studies Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-11 Kollin W. Rott, Gert Bronfort, Haitao Chu, Jared D. Huling, Brent Leininger, Mohammad Hassan Murad, Zhen Wang, James S. Hodges
Meta-analysis is commonly used to combine results from multiple clinical trials, but traditional meta-analysis methods do not refer explicitly to a population of individuals to whom the results apply and it is not clear how to use their results to assess a treatment's effect for a population of interest. We describe recently-introduced causally interpretable meta-analysis methods and apply their treatment
-
A mapping exercise using automated techniques to develop a search strategy to identify systematic review tools Res. Synth. Methods (IF 9.8) Pub Date : 2023-09-05 Anthea Sutton, Hannah O'Keefe, Eugenie Evelynne Johnson, Christopher Marshall
The Systematic Review Toolbox aims provide a web-based catalogue of tools that support various tasks within the systematic review and wider evidence synthesis process. Identifying publications surrounding specific systematic review tools is currently challenging, leading to a high screening burden for few eligible records. We aimed to develop a search strategy that could be regularly and automatically
-
DTAmetasa: An R shiny application for meta-analysis of diagnostic test accuracy and sensitivity analysis of publication bias Res. Synth. Methods (IF 9.8) Pub Date : 2023-08-28 Shosuke Mizutani, Yi Zhou, Yu-Shi Tian, Tatsuya Takagi, Tadayasu Ohkubo, Satoshi Hattori
Meta-analysis of diagnostic test accuracy (DTA) is a powerful statistical method for synthesizing and evaluating the diagnostic capacity of medical tests and has been extensively used by clinical physicians and healthcare decision-makers. However, publication bias (PB) threatens the validity of meta-analysis of DTA. Some statistical methods have been developed to deal with PB in meta-analysis of DTA
-
Two-stage or not two-stage? That is the question for IPD meta-analysis projects Res. Synth. Methods (IF 9.8) Pub Date : 2023-08-22 Richard D. Riley, Joie Ensor, Miriam Hattle, Katerina Papadimitropoulou, Tim P. Morris
Individual participant data meta-analysis (IPDMA) projects obtain, check, harmonise and synthesise raw data from multiple studies. When undertaking the meta-analysis, researchers must decide between a two-stage or a one-stage approach. In a two-stage approach, the IPD are first analysed separately within each study to obtain aggregate data (e.g., treatment effect estimates and standard errors); then
-
Rare events meta-analysis using the Bayesian beta-binomial model Res. Synth. Methods (IF 9.8) Pub Date : 2023-08-22 Katrin Jansen, Heinz Holling
In meta-analyses of rare events, it can be challenging to obtain a reliable estimate of the pooled effect, in particular when the meta-analysis is based on a small number of studies. Recent simulation studies have shown that the beta-binomial model is a promising candidate in this situation, but have thus far only investigated its performance in a frequentist framework. In this study, we aim to make
-
Advice for improving the reproducibility of data extraction in meta-analysis Res. Synth. Methods (IF 9.8) Pub Date : 2023-08-11 Edward R. Ivimey-Cook, Daniel W. A. Noble, Shinichi Nakagawa, Marc J. Lajeunesse, Joel L. Pick
Extracting data from studies is the norm in meta-analyses, enabling researchers to generate effect sizes when raw data are otherwise not available. While there has been a general push for increased reproducibility in meta-analysis, the transparency and reproducibility of the data extraction phase is still lagging behind. Unfortunately, there is little guidance of how to make this process more transparent
-
Combining meta-analysis with multiple imputation for one-step, privacy-protecting estimation of causal treatment effects in multi-site studies Res. Synth. Methods (IF 9.8) Pub Date : 2023-08-01 Di Shu, Xiaojuan Li, Qoua Her, Jenna Wong, Dongdong Li, Rui Wang, Sengwee Toh
Missing data complicates statistical analyses in multi-site studies, especially when it is not feasible to centrally pool individual-level data across sites. We combined meta-analysis with within-site multiple imputation for one-step estimation of the average causal effect (ACE) of a target population comprised of all individuals from all data-contributing sites within a multi-site distributed data
-
Inclusive critical appraisal of qualitative and quantitative findings in evidence synthesis Res. Synth. Methods (IF 9.8) Pub Date : 2023-07-31 Ajima Olaghere, David B. Wilson, Catherine Kimbrell
A diversity of approaches for critically appraising qualitative and quantitative evidence exist and emphasize different aspects. These approaches lack clear processes to facilitate rating the overall quality of the evidence for aggregated findings that combine qualitative and quantitative evidence. We draw on a meta-aggregation of implementation and process evaluations to illustrate a method for critically
-
Estimating pairwise overlap in umbrella reviews: Considerations for using the corrected covered area (CCA) index methodology Res. Synth. Methods (IF 9.8) Pub Date : 2023-07-27 Mariam Kirvalidze, Ahmad Abbadi, Lena Dahlberg, Lawrence B. Sacco, Amaia Calderón-Larrañaga, Lucas Morin
Umbrella reviews (reviews of systematic reviews) are increasingly used to synthesize findings from systematic reviews. One important challenge when pooling data from several systematic reviews is publication overlap, that is, the same primary publications being included in multiple reviews. Pieper et al. have proposed using the corrected covered area (CCA) index to quantify the degree of overlap between
-
Evaluation of the interim Cochrane rapid review methods guidance—A mixed-methods study on the understanding of and adherence to the guidance Res. Synth. Methods (IF 9.8) Pub Date : 2023-07-21 Ursula Griebler, Andreea Dobrescu, Dominic Ledinger, Pauline Klingenstein, Isolde Sommer, Robert Emprechtinger, Emma Persad, Arianna Gadinger, Marialena Trivella, Irma Klerings, Barbara Nussbaumer-Streit
The Cochrane Rapid Review Methods Group (RRMG) first released interim guidance in March 2020 to support authors in conducting rapid reviews (RRs). The objective of this mixed-methods study was to assess the adherence and investigate authors' understanding of the RRMG guidance. We identified all documents citing the Interim Cochrane RRMG guidance up to February 17, 2022 and performed an exploratory
-
Critical appraisal tools used in systematic reviews of in vitro cell culture studies: A methodological study Res. Synth. Methods (IF 9.8) Pub Date : 2023-07-18 Victor Paiva Barbosa, Bruna Bastos Silveira, Juliana Amorim dos Santos, Mylene Martins Monteiro, Ricardo D. Coletta, Graziela De Luca Canto, Cristine Miron Stefani, Eliete Neves Silva Guerra
Systematic reviews (SRs) of preclinical studies are marked with poor methodological quality. In vitro studies lack assessment tools to improve the quality of preclinical research. This methodological study aimed to identify, collect, and analyze SRs based on cell culture studies to highlight the current appraisal tools utilized to support the development of a validated critical appraisal tool for cell
-
Accounting for time dependency in meta-analyses of concordance probability estimates Res. Synth. Methods (IF 9.8) Pub Date : 2023-07-10 Matthias Schmid, Tim Friede, Nadja Klein, Leonie Weinhold
Recent years have seen the development of many novel scoring tools for disease prognosis and prediction. To become accepted for use in clinical applications, these tools have to be validated on external data. In practice, validation is often hampered by logistical issues, resulting in multiple small-sized validation studies. It is therefore necessary to synthesize the results of these studies using