Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

No evidence that mandatory open data policies increase error correction

Nature Ecology & Evolution volume 6, pages 1630–1633 (2022)Cite this article

Subjects

Matters Arising to this article was published on 31 July 2023

Abstract

Using a database of open data policies for 199 journals in ecology and evolution, we found no detectable link between data sharing requirements and article retractions or corrections. Despite the potential for open data to facilitate error detection, poorly archived datasets, the absence of open code and the stigma associated with correcting or retracting articles probably stymie error correction. Requiring code alongside data and destigmatizing error correction among authors and journal editors could increase the effectiveness of open data policies at helping science self-correct.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: The number of retractions per year before and after journals in E&E implemented open data policies.
Fig. 2: The number of article corrections per year before and after journals in E&E implemented policies mandating open data.

Similar content being viewed by others

Data availability

The data to reproduce the results of this study are available on the Open Science Framework (https://doi.org/10.17605/OSF.IO/8BRYS) and were shared with the editor and reviewers on submission.

Code availability

The code to reproduce the results of this study are available on the Open Science Framework (https://doi.org/10.17605/OSF.IO/8BRYS) and were shared with the editor and reviewers on submission.

References

  1. Hardwicke, T. E. et al. Analytic reproducibility in articles receiving open data badges at the journal Psychological Science: an observational study. R. Soc. Open Sci. 8, 201494 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  2. Enserink, M. Sea of doubts. Science 372, 560–565 (2021).

    Article  CAS  PubMed  Google Scholar 

  3. Buxton, R. T. et al. Avoiding wasted research resources in conservation science. Conserv. Sci. Pract. 3, 1–11 (2021).

    Google Scholar 

  4. Tai, T. C. & Robinson, J. P. W. Enhancing climate change research with open science. Front. Environ. Sci. 6, 1–5 (2018).

    Article  Google Scholar 

  5. Popkin, G. Data sharing and how it can benefit your scientific career. Nature 569, 445–447 (2019).

    Article  CAS  PubMed  Google Scholar 

  6. Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data 3, 160018 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  7. Roche, D. G. et al. Slow improvement to the archiving quality of open datasets shared by researchers in ecology and evolution. Proc. R. Soc. B Biol. Sci. 289, 20212780 (2022).

    Article  Google Scholar 

  8. Tedersoo, L., Küngas, R., Oras, E., Köster, K. & Helen, E. Data sharing practices and data availability upon request differ across scientific disciplines. Sci. Data 8, 1–11 (2021).

    Article  Google Scholar 

  9. Christian, T. M., Gooch, A., Vision, T. & Hull, E. Journal data policies: exploring how the understanding of editors and authors corresponds to the policies themselves. PLoS ONE 15, 1–15 (2020).

    Article  Google Scholar 

  10. Sholler, D., Ram, K., Boettiger, C. & Katz, D. S. Enforcing public data archiving policies in academic publishing: a study of ecology journals. Big Data Soc. 6, 1–18 (2019).

    Article  Google Scholar 

  11. Postma, E., Gonzalez‐Voyer, A. & Holman, L. A comment on The adaptive value of gluttony: predators mediate the life history trade‐offs of satiation threshold by Pruitt & Krauel (2010). J. Evol. Biol. 34, 1989–1993 (2021).

    Article  PubMed  Google Scholar 

  12. Stodden, V., Seiler, J. & Ma, Z. An empirical analysis of journal policy effectiveness for computational reproducibility. Proc. Natl Acad. Sci. USA 115, 2584–2589 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Rohrer, J. M. et al. Putting the self in self-correction: findings from the Loss-of-Confidence Project. Perspect. Psychol. Sci. 16, 1255–1269 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  14. Vazire, S. A toast to the error detectors. Nature 577, 9 (2020).

    Article  CAS  PubMed  Google Scholar 

  15. Culina, A., van den Berg, I., Evans, S. & Sánchez-Tójar, A. Low availability of code in ecology: a call for urgent action. PLoS Biol. 18, e3000763 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Roche, D. G. et al. Paths towards greater consensus building in experimental biology. J. Exp. Biol. 225, jeb243559 (2022).

    Article  PubMed  Google Scholar 

  17. Laurinavichyute, A., Yadav, H. & Vasishth, S. Share the code, not just the data: a case study of the reproducibility of articles published in the Journal of Memory and Language under the open data policy. J. Mem. Lang. 125, 104332 (2022).

    Article  Google Scholar 

  18. Roche, D. G., Kruuk, L. E. B., Lanfear, R. & Binning, S. A. Public data archiving in ecology and evolution: how well are we doing? PLoS Biol. 13, e1002295 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  19. Besançon, L., Bik, E., Heathers, J. & Meyerowitz-Katz, G. Correction of scientific literature: too little, too late! PLoS Biol. 20, e3001572 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  20. Holmes, N. P. I critiqued my past papers on social media—here’s what I learnt. Nature 595, 333 (2021).

    Article  CAS  PubMed  Google Scholar 

  21. Teixeira da Silva, J. A. & Al-Khatib, A. Ending the retraction stigma: encouraging the reporting of errors in the biomedical record. Res. Ethics 17, 251–259 (2021).

    Article  Google Scholar 

  22. Minocher, R., Atmaca, S., Bavero, C., McElreath, R. & Beheim, B. Estimating the reproducibility of social learning research published between 1955 and 2018. R. Soc. Open Sci. 8, 210450 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  23. Montgomerie, R. From the Editor’s desk of The American Naturalist: data transparency 2020. The American Naturalist http://comments.amnat.org/2021/01/note-since-fall-2020-robert-montgomerie.html (2021).

  24. R Project. R version 4.0.3 https://cran.r-project.org/bin/windows/base/old/4.0.3/ (2020).

Download references

Acknowledgements

We thank S. A. Binning, F. Dhane and F. Lauzon for assistance with this project, as well as J. Towse and T. Vines for helpful comments on the manuscript. We acknowledge funding by the Natural Sciences and Engineering Research Council of Canada (grant no. UIF-537860–2018 to DGR) and the European Union’s Horizon 2020 research and innovation programme under Marie Skłodowska-Curie grant agreement no. 838237-OPTIMISE to DGR.

Author information

Authors and Affiliations

  1. Department of Biology, Carleton University, Ottawa, Canada

    Ilias Berberi & Dominique G. Roche

  2. Institut de Biologie, Université de Neuchâtel, Neuchâtel, Switzerland

    Dominique G. Roche

Authors
  1. Ilias Berberi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dominique G. Roche
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

I.B. and D.G.R. conceived the study, collected and analysed the data, and wrote the manuscript.

Corresponding author

Correspondence to Ilias Berberi.

Ethics declarations

Competing interests

D.G.R. is a member of Research Data Canada’s Policy Committee, the Canadian National Committee for CODATA and the Canadian Institute for Ecology and Evolution’s Living Data Project, and the president of the Society for Open, Reliable and Transparent Ecology and Evolutionary Biology (www.sortee.org). I.B. declares no competing interests.

Peer review

Peer review information

Nature Ecology & Evolution thanks Bonnie Wintle and Felix Schönbrodt for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 The impact factor and open data policy of journals (n = 199) that publish research in ecology and evolution.

Journals were categorized into four tiers based on their data policy requirements: no data policy, recommended open data, mandatory data availability statement, and mandatory open data. See Extended Data Table 1 for descriptive and inferential statistics.

Extended Data Table 1 A) Mean journal impact factor (JIF) for four open data policy tiers of journals in ecology and evolution, and B) Post-hoc least squares means comparison of journal impact factor across four open data policy tiers A) A linear model of log-transformed JIF [log(JIF) ~ tier] revealed differences among tiers (F3,192 = 18.48, p < 0.001, R2 = 0.21). Three journals in Tier 1 did not report 2020 JIF values and were excluded from the model. SD = standard deviation. N = number of journals. B) Estimates (± SE), t-ratios, and p-values are reported for each comparison of tiers (two-sided test). Results are presented on a log-scale and adjusted for multiple comparisons by Tukey’s HSD (two-sided)
Full size table
Extended Data Table 2 Mean difference in the number of article retractions and corrections per year before and after the implementation of open data policies by journals in ecology and evolution. Presented are the number of retractions per year before and after the date of open data policy implementation (mean ± SD), the difference between both values (mean ± SD), journal sample sizes (N), t-statistics, and p-values of paired permutation tests performed for the different open data policy tiers. Each permutation test (two-sided) was performed with 10,000 iterations
Full size table
Extended Data Table 3 Total number of retractions before and after open data policies were implemented in each of four open data policy tiers of journals in ecology and evolution. For comparative purposes, we assigned the mean implementation date of journals for which information was available (that is, 2016) for journals with no data policy (a) and journals that recommend open data (b) (see Methods)
Full size table

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Berberi, I., Roche, D.G. No evidence that mandatory open data policies increase error correction. Nat Ecol Evol 6, 1630–1633 (2022). https://doi.org/10.1038/s41559-022-01879-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41559-022-01879-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing Anthropocene

Sign up for the Nature Briefing: Anthropocene newsletter — what matters in anthropocene research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: Anthropocene