Abstract
In various fields, scientific article publication is a measure of productivity and in many occasions it is used as a critical factor for evaluating researchers. Therefore, a lot of time is dedicated to writing articles that are then submitted for publication in journals. Nevertheless, the publication process in general and the review process in particular tend to be rather slow. This is the case for instance of computer science (CS) journals. Moreover, the process typically lacks in transparency, where information about the duration of the review process is at best provided in an aggregated manner, if made available at all. In this paper, we develop a framework as a step towards bringing more reliable data with respect to review duration. Based on this framework, we implement a tool—journal response time (JRT), that allows for automatically extracting the review process data and helps researchers to find the average response times of journals, which can be used to study the duration of CS journals’ peer review process. The information is extracted as metadata from the published articles, when available. This study reveals that the response times publicly provided by publishers differ from the actual values obtained by JRT (e.g., for ten selected journals the average duration reported by publishers deviates by more than 500% from the actual average value calculated from the data inside the articles), which we suspect could be from the fact that, when calculating the aggregated values, publishers consider the review time of rejected articles too (including quick desk-rejections that do not require reviewers).
Similar content being viewed by others
Notes
E-prints posted in http://www.arxiv.org are not peer reviewed.
The full list of the journals for each publisher can be found here: http://dtim.essi.upc.edu/jrt/journals.php.
References
Alberts, B., Hanson, B., & Kelner, K. L. (2008). Reviewing peer review. Science, 321(5885), 15.
Ausloos, M., Nedic, O., & Dekanski, A. (2019). Seasonal entropy, diversity and inequality measures of submitted and accepted papers distributions in peer-reviewed journals. Entropy, 21(6), 564–575.
Azar, O.(2003). Rejections and the importance of first response times (or: How many rejections do others receive?). General Economics and Teaching 0309002, University Library of Munich, Germany.
Barbero, E. J. (2008). Journal paper requirement for PhD graduation. Journal of Engineering Education, 2(2), 91–102.
Bjork, B.-C., & Solomon, D. (2013). The publishing delay in scholarly peer-reviewed journals. Journal of Informetrics, 7(4), 914–923.
Buttler, D. (2013). The dark side of publishing. Nature, 495(7442), 433–435.
Daraio, C., Lenzerini, M., Leporelli, C., Moed, H. F., Naggar, P., Bonaccorsi, A., et al. (2016). Data integration for research and innovation policy: An ontology-based data management approach. Scientometrics, 106(2), 857–871.
Daraio, C., Lenzerini, M., Leporelli, C., Naggar, P., Bonaccorsi, A., & Bartolucci, A. (2016). The advantages of an Ontology-Based Data Management approach: Openness, interoperability and data quality. Scientometrics, 108(1), 441–455.
Doi, S. A. R., Salzman-Scott, S. A., & Onitilo, A. A. (2016). Validation of the CoRE questionnaire for a medical journal peer review. Accountability in Research, 23(1), 47–52.
Drvenica, I., Bravo, G., Vejmelka, L., Dekanski, A., & Nedic, O. (2018). Peer review of reviewers: The author’s perspective. Publications, 7(1), 1.
Ellison, G. (2002). Evolving standards for academic publishing: A q-r theory. Journal of Political Economy, 110(5), 994–1034.
Ellison, G. (2007). Is peer review in decline? Economic Inquiry, 49(3), 635–657.
Huisman, J., & Smits, J. (2017). Duration and quality of the peer review process: The author’s perspective. Scientometrics, 113(1), 633–650.
Jennings, C. G. (2020). Quality and value: The true purpose of peer review? http://blogs.nature.com/peer-to-peer/2006/06/quality_and_value_the_true_pur.html. Accessed September 20, 2020.
Kareiva, P., Marvier, M., West, S., & Hornisher, J. (2002). Slow-moving journals hinder conservation efforts. Nature, 420, 15.
Lewin, A. Y. (2014). The peer-review process: The good, the bad, the ugly, and the extraordinary. Management and Organization Review, 10(2), 167–173.
Lotriet Cornelius, J. (2012). Reviewing the review process: Identifying sources of delay. The Australasian Medical Journal, 5, 26–29.
Mrowinski, M. J., Fronczak, A., Fronczak, P., Nedic, O., & Ausloos, M. (2016). Review time in peer review: Quantitative analysis and modelling of editorial workflows. Scientometrics, 107(1), 271–286.
Mrowinski, M. J., Fronczak, A., Fronczak, P., Nedic, O., & Dekanski, A. (2020). The hurdles of academic publishing from the perspective of journal editors: A case study. Scientometrics, 125(1), 115–133.
Peacock, S. (2017). The PhD by publication. International Journal of Doctoral Studies, 12, 123–135.
Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195.
Resnik, D., Gutierrez-Ford, C., & Peddada, S. (2008). Perceptions of ethical problems with scientific journal peer review: An exploratory study. Science and Engineering Ethics, 14(3), 305–310.
Robins, L., & Kanowski, P. (2008). PhD by publication: A student’s perspective. Journal of Research Practice, 4(2), 3.
Smith, A. J. (1990). The task of the referee. Computer, 23(4), 65–71.
Solomon, D. J., & Björk, B.-C. (2012). Publication fees in open access publishing: Sources of funding and factors influencing choice of journal. Journal of the American Society for Information Science and Technology, 63(1), 98–107.
Tite, L., & Sara, S. (2007). Why do peer reviewers decline to review? A survey. Journal of Epidemiology and Community Health, 61(1), 9–12.
Ware, M., & Mabe, M. (2015). The STM Report: An overview of scientific and scholarly journal publishing. 03 2015.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bilalli, B., Munir, R.F. & Abelló, A. A framework for assessing the peer review duration of journals: case study in computer science. Scientometrics 126, 545–563 (2021). https://doi.org/10.1007/s11192-020-03742-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-020-03742-9