Skip to main content
Download PDF
  • Study protocol
  • Open access
  • Published:

Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews

Research Integrity and Peer Review volume 4, Article number: 27 (2019) Cite this article

Abstract

Background

Titles and abstracts are the most read sections of biomedical papers. It is therefore important that abstracts transparently report both the beneficial and adverse effects of health care interventions and do not mislead the reader. Misleading reporting, interpretation, or extrapolation of study results is called “spin”. In this study, we will assess whether adverse effects of orthodontic interventions were reported or considered in the abstracts of both Cochrane and non-Cochrane reviews and whether spin was identified and what type of spin.

Methods

Eligibility criteria were defined for the type of study designs, participants, interventions, outcomes, and settings. We will include systematic reviews of clinical orthodontic interventions published in the five leading orthodontic journals and in the Cochrane Database. Empty reviews will be excluded. We will manually search eligible reviews published between 1 August 2009 and 31 July 2019. Data collection forms were developed a priori. All study selection and data extraction procedures will be conducted by two reviewers independently. Our main outcomes will be the prevalence of reported or considered adverse effects of orthodontic interventions in the abstract of systematic reviews and the prevalence of “spin” related to these adverse effects. We will also record the prevalence of three subtypes of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation-related spin. All statistics will be calculated for the following groups: (1) all journals individually, (2) all journals together, and (3) the five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately. Generalized linear models will be developed to compare the various groups.

Discussion

We expect that our results will raise the awareness of the importance of reporting and considering of adverse effects and the presence of the phenomenon of spin related to these effects in abstracts of systematic reviews of orthodontic interventions. This is important, because an incomplete and inadequate reporting, interpretation, or extrapolation of findings on adverse effects in abstracts of systematic reviews can mislead readers and could lead to inadequate clinical practice. Our findings could result in policy implications for making judgments about the acceptance for publication of systematic reviews of orthodontic interventions.

Peer Review reports

Background

Readers of the biomedical literature mostly just screen the title and the abstract of an article without assessing the full publication [1]. The beneficial and adverse effects of interventions should therefore be transparently reported in these summaries and should not mislead its readers. Misleading reporting, interpretation, or extrapolation of study results is called “spin” [2,3,4]. We will assess in abstracts of both Cochrane and non-Cochrane reviews whether adverse effects of orthodontic interventions were reported or considered and whether spin was identified and what type of spin.

Titles and abstracts are the most read sections of biomedical papers [1], because assessing the full research article is often conditioned by paywalls or because of a lack of time or language issues of the readers [1]. Abstracts should therefore clearly and truthfully reflect the objectives, methods, results, and the interpretation of research findings. The standard for Methodological Expectations of Cochrane Intervention Reviews (MECIR) [5] has listed a series of highly desirable and mandatory items that should be consulted by reviewers when preparing the abstract of their reviews. Item R13 of the MECIR standard states that: “The Abstract of the review should aim to reflect a balanced summary of the benefits and harms of the intervention.” This mandatory item is particularly crucial for presenting adverse effects of health care interventions, because these effects are often poorly reported in systematic reviews [6]. Numerous epidemiological studies have also shown that the assessment and reporting of adverse effects of interventions in primary research studies is often suboptimal [7,8,9,10,11]. We adopted Cochrane’s definition of adverse effects: “An adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility” [12, 13]. This definition and other key terminology in this manuscript are summarized in Table 1 [2,3,4, 12,13,14,15].

Table 1 Glossary of terms
Full size table

When presenting information on adverse effects in the abstract, it is also crucial that it does not mislead the reader. A distorted presentation of study results has been defined as “spin” [3], but more elaborate definitions are also used (Table 1). The term spin was first used in 1995 in the medical literature by Horton [16] and has been further subdivided into three categories [4]: misleading reporting-related spin, misleading interpretation-related spin, and misleading extrapolation-related spin (Table 1). Yavchitz et al. [17] have ranked the various types of spin according to their severity. The severest form of spin in abstracts of systematic reviews and meta-analyses was scored for “conclusions that contain recommendations for clinical practices that were not supported by findings” [17]. A high prevalence of the various types of spin has been identified in multiple epidemiological studies [4, 18,19,20,21,22]. Boutron et al. [18] found spin in 50% (36/72) of the conclusions sections of the main text of parallel-group RCTs and in 58.3% (42/72) of the conclusions sections of the abstracts. Spin was also common in diagnostic accuracy studies published in journals with high impact factors [22]. Lockyer et al. [21] showed that spin is a frequent phenomenon in abstracts of RCTs of wound treatments, and Lazarus et al. [4] identified at least one example of spin in 84% (107/128) of the abstracts of non-randomized intervention studies. Spin is in strong conflict with the Declaration of Helsinki [23] that states that: “Authors have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports.”

In this study, we will assess whether potential adverse effects of orthodontic interventions were reported or considered (i.e., discussed, weighed, etc.) in the abstract of systematic reviews. We will further assess whether spin was introduced regarding information on these adverse effects in the abstract, and we will categorize the types of spin (Table 1). We will assess these issues in the five leading orthodontic journals and those included in the Cochrane Database of Systematic Reviews. In these reviews, we will assess adverse effects such as pain as a result of tooth movement and the adverse effects defined by Preoteasa et al. (Table 2) [24]. Scoping searches in the orthodontic literature confirmed the knowledge gaps on our research questions. Our pilot studies quantified these gaps and confirmed the need to address these questions. We will assess these issues in systematic reviews, because they are increasingly consulted by patients [25] and when well-conducted systematic reviews are considered among the information sources with the highest level of evidence [26]. Our research questions are important, because incomplete or misleading information on adverse effects of interventions may have detrimental effects on the treatment of orthodontic patients.

Table 2 Adverse effects hypothetically linked to orthodontic interventions [24]
Full size table

Objectives

The objectives of this research study are summarized in the following research questions:

Research questions

  • In abstracts of systematic reviews of orthodontic interventions, were potential adverse effects of these interventions reported or considered (i.e., discussed, weighed, etc.)?

  • In abstracts of systematic reviews of orthodontic interventions, was spin identified in the reporting, interpretation, and extrapolation of adverse effects?

  • What was the prevalence of each type of spin?

Methods

This protocol is reported according to the guidance of the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement, and the PRISMA-P checklist is included as Additional file 1 [27, 28]. We adopted the same flow of research methods as reported in our published protocol on seeking adverse effects in systematic reviews of orthodontic interventions (Fig. 1) and conducted our pilot tests on the same sample of systematic reviews as was described in our previous protocol [15]. Our sample size of 14 reviews for the pilot test was calculated with the formula reported by Viechtbauer et al. [29]. Further details on the methods of our pilot test are reported in Additional file 2. This pilot test found that the reviewers in only 35.7% (5/14) of the abstracts reported or considered (i.e., discussed, weighed) potential adverse effects of orthodontic interventions. This sample identified an overall prevalence of 14.3% (2/14) of spin in the abstract on adverse effects of orthodontic interventions. Both cases of spin were “misleading reporting-related spin.” The following sections describe our planned methods based on these pilot tests. We will not start the selection of eligible reviews and data extraction prior to the complete acceptance of this protocol for publication.

Fig. 1
figure 1

Flow diagram of the research methods. “Flow diagram of the research methods” was published previously by Steegmans et al. [15] in the journal “Systematic Reviews,” which is an open access journal of BioMed Central. Copyright on any open access article in a journal published by BioMed Central is retained by the author(s)

Full size image

Eligibility criteria

We will adopt the same eligibility criteria that were defined for our published protocol on seeking adverse effects in systematic reviews of orthodontic interventions [15]. To avoid misinterpretation, we copied and pasted these eligibility criteria into Table 3 [15, 30].

Table 3 Eligibility criteria
Full size table

Information sources

We will manually search the Cochrane library [14] and the websites of the five leading orthodontic journals to identify eligible systematic reviews published between 1 August 2009 and 31 July 2019. We chose this starting date because the first of August 2009 coincides with the launch of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement and its guidance paper on 21 July 2009 [27, 28]. Journal selection was based on two criteria: (1) the journal has been published for 10 years or more and (2) the impact factor. The journal citation reports by Clarivate Analytics were consulted to identify the five leading orthodontic journals based on impact factor [31]. The following five orthodontic journals fulfilled both criteria: European Journal of Orthodontics (EJO), American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), Angle Orthodontist, The Korean Journal of Orthodontics, and Orthodontics and Craniofacial Research.

Study records

Data management

  • Two authors (PS and RMR) will conduct all study selection and data extraction procedures independently.

  • Pilot tests were conducted to train both reviewers in applying these methods consistently and for calibration purposes [28].

  • We will apply the following strategies in the case of disagreement between the two authors on the eligibility of a paper or the extraction of data: (1) discussions between reviewers, (2) rereading the paper, (3) or if necessary contacting its authors [32]. Persistent disagreements will be resolved through the consultation and arbitration of a methodologist (NDG).

  • All eligible systematic reviews will be downloaded, and all extracted data will be collected in an Excel spreadsheet.

Selection process

  • All titles and abstracts in the websites of the five orthodontic journals will be hand-searched to identify eligible reviews. The section “Dentistry and Oral health” will be searched in the Cochrane library for eligible Cochrane reviews [14].

  • We will only include the latest version of a review when updates have been published.

  • Authors will be contacted in the case of doubt regarding multiple publications of the same review. We plan to include the first publication, but will make this decision on a case by case basis and will report the rationale for this choice.

  • Our selection procedures will be presented in a PRISMA flow diagram [32, 33].

  • All included and excluded studies will be presented in tables, and the rationale for exclusion will be given for each excluded review.

Data collection process

  • All eligible studies together with their supplemental files will be merged into binder PDFs, and pertinent search terms are linked to these documents to facilitate data extraction [34, 35].

  • Eligible search terms were identified through searches in thesauri and in key articles on adverse effects [13, 36,37,38,39,40]. These terms are given in Additional file 3 and are identical to those used in our protocol on seeking adverse effects in systematic reviews of orthodontic interventions [15].

  • Our pilot-tested data collection forms will be used for all data extraction procedures (Additional file 4). The PRISMA [32, 33] and the PRISMA-P [27, 28] checklists and guidance were consulted to develop these forms. The criteria for scoring pertinent data items are defined in these forms.

  • The entire eligible review of both orthodontic and Cochrane reviews will be searched for data items, i.e., the text, tables, figures, and all supplemental files. The plain language summary in the eligible Cochrane reviews will not be searched for data items.

  • When during the data extraction procedure changes are made in the data collection forms, we will present this with rationale in the section “Differences between the protocol and review.”

Scoring adverse effects of orthodontic interventions

  • We will use the framework of categories of known orthodontic adverse effects as defined by Preoteasa et al. [24] (Table 2). We will also include pain as a result of tooth movement as an adverse effect. Potential adverse effects that are identified during data extraction will be discussed between the two reviewers (PS and RMR). We will report the rationale when including additional adverse effects and will categorize them.

  • Ambiguous outcomes that could be interpreted as either a beneficial or an adverse outcome will not be scored as “adverse.” The rationale for this score will be given. We will only consider ambiguous outcomes as “adverse” when the review authors define these outcomes as such and make a strong case for this classification.

Scoring spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions

  • We will assess three types of spin, i.e., misleading reporting, misleading interpretation, and misleading extrapolation on adverse effects of orthodontic interventions in the abstract (Table 4). Each type of spin will be assessed separately for reviews that either did or did not seek adverse effects of interventions.

  • To facilitate our scoring procedures and to reduce the risk of misinterpretation, we subdivided each type of spin into categories and defined each category (Table 4). We will score the presence of spin when spin is identified for one or more of these categories. The scoring procedures are summarized in Additional file 4. Pilot tests were conducted to assess the validity of these procedures.

Table 4 Types of spin in reviews that did or did not seek adverse effects of interventions
Full size table

Outcomes and statistical analyses

  • Figure 2 a and b present all research questions in a flow diagram, and Table 5 lists all planned outcomes.

  • We will calculate and report all prevalence data with their 95% confidence levels.

  • We calculate the prevalence statistics for (1) all journals as one group, (2) the group of five leading orthodontic journals and the Cochrane Database of Systematic Reviews separately, and (3) each individual journal separately. Generalized linear models will be developed having the following outcomes for the abstracts of systematic reviews of orthodontic interventions: the reporting or considering of potential adverse effects of interventions/no reporting or considering of potential adverse effects of interventions (binary); presence of SPIN/absence of “SPIN” (binary); and misleading reporting/misleading interpretation/misleading extrapolation/no SPIN (categorical). The models will account for journal category (Cochrane Database of Systematic Reviews vs others), individual journals, and the geographical location of the study. Statistical significance will be based on a p value < 0.05. Stata software (Stata Corporation, College Station, TX, USA) version 15 will be used for all the statistical analyses [41].

  • All outcomes that will be introduced or eliminated post hoc will be reported together with the rationale for inclusion or exclusion.

Fig. 2
figure 2

a Reporting or considering adverse effects of orthodontic interventions in the abstract. b Spin on adverse effects of orthodontics in the abstract

Full size image
Table 5 Summary of findings
Full size table

Reporting of the research study and data management

  • The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement will be used as the guideline for reporting the completed cross-sectional study [42].

  • A data management plan was prepared for the long-term storage of our research data [43] in the case that the publisher of our completed research study will not or will only partly store our raw data. We consulted the Registry of Research Data Repositories [44] to identify an appropriate repository for our type of research data. We selected Dryad [45] for two reasons: (1) it is an international repository of data of peer-reviewed scientific and medical research and (2) it also includes data sets for which no specific data repository exist such as meta-epidemiological research data of systematic reviews in orthodontics. Our data management plan implies that (1) all our research data will be made freely available, (2) our completed article will present a link to a repository in which all raw data of the study will be deposited, (3) the repository is registered in the Registry of Research Data Repositories [44], (4) our research data will be reported in a format that permits other researchers to understand, cite, and reuse these data, (5) all sensitive data will be protected, and (6) it will be reassessed frequently and also updated if necessary [43, 44].

Differences between the protocol and the completed study

  • We will report all modifications between the protocol and the final research study. The rationale for each of these changes will be given.

  • We will also report the consequences of these modifications on the magnitude, direction, and the validity of the outcomes [46].

Discussion

Strengths

Key strengths of this research study include the following: (1) we conducted extensive scoping searches and pilot studies to fine-tune our research questions and methods. These activities confirmed the importance of our questions. (2) Our research team consists of two topic experts (PS and RMR) and two methodologists (RMR and NDG). (3) All study selection and data collection procedures will be undertaken independently by two authors (PS and RMR). Calibration of these operators was done during the pilot studies. (4) To guarantee reproducibility and full access to our data, we will publish our protocol a priori and will include all raw data of the completed research study in additional files or will deposit them in an open-access repository [43,44,45, 47].

Limitations

Including only orthodontic intervention reviews published in the five leading orthodontic journals and in the Cochrane Database of Systematic Reviews could be a limitation, but we expect that the findings in this subgroup of journals will underestimate the true severity of spin on adverse effects of interventions in the abstracts of these reviews. Including only reviews published in the last 10 years could also be a limitation. However, we chose this period because it brings the current knowledge status on our research questions to the foreground and these 10 years coincide with the launch in 2009 of the checklist of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [32, 33].

Importance and beneficiaries

In this research study, we will address three key questions in abstracts of systematic reviews of orthodontic interventions: whether potential adverse effects of these interventions were reported or considered, whether spin was identified regarding information on these adverse effects, and the type of spin. These issues are important, because (1) the assessment and reporting of adverse effects of interventions is often suboptimal [7,8,9,10,11], (2) titles and abstracts are the most read sections of papers in the biomedical literature [1], (3) a high prevalence of spin has been identified in abstracts of both randomized and non-randomized studies [4, 21], and (4) incomplete or inadequate reporting, interpretation, or extrapolation of findings on adverse effects in the abstract can mislead readers and could lead to inadequate practice [4]. Our results will raise the awareness of considering adverse effects and the phenomenon of spin regarding these effects in abstracts of systematic reviews of orthodontic interventions. Patients, clinicians, researchers, editors, peer-reviewers, guideline developers, policy makers, and research funders will all be beneficiaries of the findings of this research study.

Availability of data and materials

Not applicable

Abbreviations

MECIR:

Methodological Expectations of Cochrane Intervention Reviews

PRISMA:

Preferred reporting items for systematic reviews and meta-analyses

PRISMA-P:

Preferred Reporting Items for Systematic review and Meta-Analysis-Protocols

References

  1. Dogan RI, Murray GC, Névéol A, Lu Z. Understanding PubMed user search behavior through log analysis. Database (Oxford). 2009;2009:bap018 Epub 2009 Nov 27.

    Google Scholar 

  2. Boutron I, Ravaud P. Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018;115(11):2613–9. https://doi.org/10.1073/pnas.1710755115.

    Article  Google Scholar 

  3. Haneef R, Yavchitz A, Ravaud P, Baron G, Oranksy I, Schwitzer G, Boutron I. Interpretation of health news items reported with or without spin: protocol for a prospective meta-analysis of 16 randomised controlled trials. BMJ Open. 2017;7(11):e017425. https://doi.org/10.1136/bmjopen-2017-017425.

    Article  Google Scholar 

  4. Lazarus C, Haneef R, Ravaud P, Boutron I. Classification and prevalence of spin in abstracts of non-randomized studies evaluating an intervention. BMC Med Res Methodol. 2015;15:85. https://doi.org/10.1186/s12874-015-0079-x.

    Article  Google Scholar 

  5. Higgins JPT, Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR). Standards for the conduct and reporting of new Cochrane Intervention Reviews, reporting of protocols and the planning, conduct and reporting of updates. Cochrane: London, 2016. Available from: https://methods.cochrane.org/sites/default/files/public/uploads/mecir_printed_booklet_final_v1.02.pdf (accessed 18 Feb 2019).

  6. Parsons R, Golder S, Watt I. More than one-third of systematic reviews did not fully report the adverse events outcome. J Clin Epidemiol. 2019;108:95–101. https://doi.org/10.1016/j.jclinepi.2018.12.007.

    Article  Google Scholar 

  7. Bagul NB, Kirkham JJ. The reporting of harms in randomized controlled trials of hypertension using the CONSORT criteria for harm reporting. Clin Exp Hypertens. 2012;34(8):548–54. https://doi.org/10.3109/10641963.2012.681724 Epub 2012 May 9.

    Article  Google Scholar 

  8. Hodkinson A, Kirkham JJ, Tudur-Smith C, Gamble C. Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open. 2013;3(9):e003436. https://doi.org/10.1136/bmjopen-2013-003436.

    Article  Google Scholar 

  9. Péron J, Maillet D, Gan HK, Chen EX, You B. Adherence to CONSORT adverse event reporting guidelines in randomized clinical trials evaluating systemic cancer therapy: a systematic review. J Clin Oncol. 2013;31(31):3957–63. https://doi.org/10.1200/JCO.2013.49.3981 Epub 2013 Sep 23.

    Article  Google Scholar 

  10. Pitrou I, Boutron I, Ahmad N, Ravaud P. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med. 2009;169(19):1756–61.

    Article  Google Scholar 

  11. Smith SM, Chang RD, Pereira A, Shah N, Gilron I, Katz NP, Lin AH, McDermott MP, Rappaport BA, Rowbotham MC, Sampaio C, Turk DC, Dworkin RH. Adherence to CONSORT harms-reporting recommendations in publications of recent analgesic clinical trials: an ACTTION systematic review. Pain. 2012;153(12):2415–21. https://doi.org/10.1016/j.pain.2012.08.009 Epub 2012 Sep 15.

    Article  Google Scholar 

  12. Glossary of terms in the Cochrane Collaboration. Version 4.2.5. Updated May 2005. Available from: http://aaz.hr/resources/pages/57/7.%20Cochrane%20glossary.pdf (accessed 27 Sept 2019).

  13. Loke YK, Price D, Herxheimer A. Chapter 14: adverse efffects. In: Higgins JPT, Green S. editors. Cochrane handbook for systematic reviews of interventions version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from: http://handbook-5-1.cochrane.org/ (accessed 29 Aug 2019).

  14. Cochrane library. Available from: https://www.cochranelibrary.com/about/about-cochrane-reviews (accessed 24 Sept 2019).

  15. Steegmans PAJ, Bipat S, Meursinge Reynders RA. Seeking adverse effects in systematic reviews of orthodontic interventions: protocol for a cross-sectional study. Syst Rev. 2019;8(1):89. https://doi.org/10.1186/s13643-019-1000-1.

    Article  Google Scholar 

  16. Horton R. The rhetoric of research. BMJ. 1995;310(6985):985–7.

    Article  Google Scholar 

  17. Yavchitz A, Ravaud P, Altman DG, Moher D, Hrobjartsson A, Lasserson T, Boutron I. A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity. J Clin Epidemiol. 2016;75:56–65. https://doi.org/10.1016/j.jclinepi.2016.01.020 Epub 2016 Feb 2.

    Article  Google Scholar 

  18. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303(20):2058–64. https://doi.org/10.1001/jama.2010.651.

    Article  Google Scholar 

  19. Haneef R, Lazarus C, Ravaud P, Yavchitz A, Boutron I. Interpretation of results of studies evaluating an intervention highlighted in Google health news: a cross-sectional study of news. PLoS One. 2015;10(10):e0140889. https://doi.org/10.1371/journal.pone.0140889 eCollection 2015.

    Article  Google Scholar 

  20. Latronico N, Metelli M, Turin M, Piva S, Rasulo FA, Minelli C. Quality of reporting of randomized controlled trials published in Intensive Care Medicine from 2001 to 2010. Intensive Care Med. 2013;39(8):1386–95. https://doi.org/10.1007/s00134-013-2947-3 Epub 2013 Jun 7.

    Article  Google Scholar 

  21. Lockyer S, Hodgson R, Dumville JC, Cullum N. “Spin” in wound care research: the reporting and interpretation of randomized controlled trials with statistically non-significant primary outcome results or unspecified primary outcomes. Trials. 2013;14:371. https://doi.org/10.1186/1745-6215-14-371.

    Article  Google Scholar 

  22. Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology. 2013;267(2):581–8. https://doi.org/10.1148/radiol.12120527 Epub 2013 Jan 29.

    Article  Google Scholar 

  23. World Medical Association Declaration of Helsinki Ethical Principles for Medical Research Involving Human Subjects. Research registration and publication and dissemination of results. Principle 36. 64th WMA General Assembly, Fortaleza, Brazil, October 2013. Published Online: October 19, 2013. doi:10.1001/jama.2013.281053. Available from: https://www.wma.net/wp-content/uploads/2016/11/DoH-Oct2013-JAMA.pdf (accessed 28 Sept 2019).

  24. Preoteasa CT, Ionescu E, Preoteasa E. Chapter 18: Risks and complications associated with orthodontic treatment. In: Bourzgui F. (editor). Orthodontics-Basic aspects and clinical considerations. March 9, 2012 under CC BY 3.0 license. www.intechopen.com. Available from: https://www.intechopen.com/books/orthodontics-basic-aspects-and-clinical-considerations/risks-and-complications-associated-with-orthodontic-treatment (accessed 29 Sept 2019).

  25. Reeves BC. Reporting of harms in systematic reviews and their primary studies. BMJ. 2014;349:g6819. https://doi.org/10.1136/bmj.g6819.

    Article  Google Scholar 

  26. Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.

    Article  Google Scholar 

  27. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1.

    Article  Google Scholar 

  28. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA, the PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.

    Article  Google Scholar 

  29. Viechtbauer W, Smits L, Kotz D, Budé L, Spigt M, Serroyen J, Crutzen R. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015;68(11):1375–9. https://doi.org/10.1016/j.jclinepi.2015.04.014 Epub 2015 Jun 6.

    Article  Google Scholar 

  30. O’Connor D, Green S, Higgins JPT (editors). Chapter 5: Defining the review question and developing criteria for including studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. [online] Available from: www.cochrane-handbook.org. (accessed 29 Sept 2019).

  31. Clarivate Analytics. Available from: https://clarivate.com/ (accessed 29th Sept 2019).

  32. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100 Epub 2009 Jul 21.

    Article  Google Scholar 

  33. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

    Article  Google Scholar 

  34. Adobe Merge PDFs, combine files into one PDF. Available from: http://www.wikihow.com/Merge-PDF-Files (accessed 29 Sept 2019).

  35. Acrobat for legal professionals. Searching and marking multiple words. [online] Available from: http://blogs.adobe.com/acrolaw/2010/04/searching-and-marking-multiple-words-in-a-pdf/ (accessed 29 Sept 2019).

  36. Ioannidis JP, Evans SJ, Gøtzsche PC, O'Neill RT, Altman DG, Schulz K, Moher D, CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141(10):781–8.

    Article  Google Scholar 

  37. Golder S, Loke YK, Zorzela L. Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. J Clin Epidemiol. 2013;66(3):253–60. https://doi.org/10.1016/j.jclinepi.2012.09.013.

    Article  Google Scholar 

  38. Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ. 2014;349:g6501. https://doi.org/10.1136/bmj.g6501.

    Article  Google Scholar 

  39. Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A, Loke Y, Vohra S. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ. 2014;348:f7668. https://doi.org/10.1136/bmj.f7668 Review.

    Article  Google Scholar 

  40. Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S, PRISMA Harms Group. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;352:i157. https://doi.org/10.1136/bmj.i157.

    Article  Google Scholar 

  41. StataCorp. Stata Statistical Software: Release 15. College Station: StataCorp LLC.; 2017.

    Google Scholar 

  42. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–7.

    Article  Google Scholar 

  43. Schiermeier Q. For the record. Making project data freely available for open science. Nature. 2018;555:403–5.

    Article  Google Scholar 

  44. Registry of Research Data Repositories. [online] Available from: https://www.re3data.org/ (accessed 29 Sept 2019).

  45. Dryad. [online] Available from: https://datadryad.org/stash (accessed 28 Sept 2019).

  46. Higgins JPT, Deeks JJ, Altman DG (editors). Chapter 16: Special topics in statistics. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. [online] Available from: http://handbook-5-1.cochrane.org/ (accessed 29 Sept 2019).

  47. Goodman SN, Fanelli D, Ioannidis JP. What does research reproducibility mean? Sci Transl Med. 2016;8(341):341ps12. https://doi.org/10.1126/scitranslmed.aaf5027.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable

Funding

All expenses for preparing this protocol and for conducting the subsequent research study will be paid evenly by each author.

Author information

Authors and Affiliations

  1. Department of Orthodontics, Academisch Centrum Tandheelkunde Amsterdam (ACTA), University of Amsterdam, Gustav Mahlerlaan 3004, 1081, LA, Amsterdam, The Netherlands

    Pauline A. J. Steegmans

  2. Center for Veterinary Health Sciences, Oklahoma State University, 2065 W Farm Road, Stillwater, OK, 74078, USA

    Nicola Di Girolamo

  3. EBMVet, Via Sigismondo Trecchi 20, 26100, Cremona, CR, Italy

    Nicola Di Girolamo

  4. Department of Oral and Maxillofacial Surgery, Academic Medical Center, University of Amsterdam, Meibergdreef 9, 1105, AZ, Amsterdam, The Netherlands

    Reint A. Meursinge Reynders

  5. Studio di Ortodonzia, Via Matteo Bandello 15, 20123, Milan, Italy

    Reint A. Meursinge Reynders

Authors
  1. Pauline A. J. Steegmans
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicola Di Girolamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Reint A. Meursinge Reynders
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PS and RMR conceived and designed the study protocol for this cross-sectional study. RMR is the guarantor. PS and RMR conducted the pilot testing of the study selection procedures and data extraction forms and fine-tuned the research protocol after the pilot testing. NDG provided support on methodological and statistical issues and assisted in the overall fine-tuning of this protocol. All three authors read and approved the final protocol.

Corresponding author

Correspondence to Reint A. Meursinge Reynders.

Ethics declarations

Ethics approval and consent to participate

Not applicable

Consent for publication

Not applicable

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

Checklist for the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015 statement.

Additional file 2.

Pilot tests.

Additional file 3.

Search terms and their derivatives.

Additional file 4.

Data collection forms.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steegmans, P.A.J., Di Girolamo, N. & Meursinge Reynders, R.A. Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews. Res Integr Peer Rev 4, 27 (2019). https://doi.org/10.1186/s41073-019-0084-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41073-019-0084-4

Keywords

Research Integrity and Peer Review

ISSN: 2058-8615

Contact us