Abstract
Data accuracy is essential for reliable and valid altmetrics analysis. Although Twitter and Facebook altmetrics data are widely used for scholarly communication and scientific evaluation, few studies have tapped into their accuracy issue. Based on content analysis of random sample records over two phases, this study has investigated and compared the accuracy of Twitter and Facebook altmetrics data. Major conclusions are drawn as follows. (1) Three error types were identified from the altmetric data provider and six error types were identified from the altmetric data aggregator. Twitter and Facebook have shared most of the error types except for minor differences in the sub-categories. (2) The overall error rate is substantially high, being 17% and 32% for Twitter and Facebook respectively in April, 2019. However, except for publication date error and posting date error, the percentage of the other error types is relatively low (being around 3%). (3) The percentage of error types related to the dynamic nature of Twitter and Facebook is increasing over time, while percentage of error types concerning the bibliographic data is decreasing over time. (4) The error types are either “high seriousness low percentage” or “low seriousness high percentage”, therefore, they would probably not bring significant negative influence. (5) Underlying reasons of these error types are various. They could be attributable to the Twitter (or Facebook) user, Twitter (or Facebook) platform, altmetric database, as well as the third-party data provider. These results suggest that Twitter and Facebook altmetrics data in the Altmetric database are reliable on the whole, although there is still space for further improvement.
Similar content being viewed by others
References
Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.
Bornmann, L., & Haunschild, R. (2018). Alternative article-level metrics: The use of alternative metrics in research evaluation. Embo Reports, 19(12), e47260.
Bornmann, L., Haunschild, R., & Adams, J. (2019). Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (ref). Journal of Informetrics, 13(1), 325–340.
Duan, Q., & Pan, X. (2017). Identification of emerging topics in science using social media. Journal of the China Society for Scientific and Technical Information, 36(12), 1216–1223. (in Chinese).
Enkhbayar, A., Haustein, S., Barata, G., & Alperin, J. P. (2020). How much research shared on facebook happens outside of public pages and groups? A comparison of public and private online activity around plos one papers. Quantitative Science Studies, 1(3), 1–22.
Fang, Z., & Costas, R. (2020). Studying the accumulation velocity of altmetric data tracked by altmetric.com. Scientometrics, 123, 1077–1101.
Fang, Z., Dudek, J., & Costas, R. (2020). The stability of Twitter metrics: A study on unavailable Twitter mentions of scientific publications. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.24344
Fraumann, G. (2017). Valuation of Altmetrics in Research Funding. (Unpublished master’s thesis). University of Tampere.
Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423.
Hawkins, C. M., Duszak, R., & Rawson, J. V. (2014). Social media in radiology: Early trends in Twitter microblogging at radiology’s largest international meeting. Journal of the American College of Radiology, 11(4), 387–390.
Ministry of Education of China. (2020). The guidelines for improving the application of SCI related indicators and establishing proper evaluation direction in higher education institutions. Retrieved from http://www.moe.gov.cn/srcsite/A16/moe_784/202002/t20200223_423334.html
NISO. (2016). Outputs of the NISO alternative assessment project. Retrieved from https://www.niso.org/publications/rp-25-2016-altmetrics
Parvin, S. (2017). Facebook as a communication tool for an academic library: East West University Library, Bangladesh Perspective. The International Information & Library Review, 49(3), 237–247.
Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013). Tweeting links to academic articles. Cybermetrics, 17(1), 1–8.
Weller, K., Bruns, A., Burgess, J., Mahrt, M., & Puschmann, C. (2014). Twitter and society. Peter Lang International Academic Publishers.
Wilkinson, S. E., Basto, M. Y., Perovic, G., Lawrentschuk, N., & Murphy, D. G. (2015). The social media revolution is changing the conference experience: Analytics and trends from eight international meetings. BJU International, 115(5), 839–846.
Wooldridge, J., & King, M. B. (2018). Altmetric scores: An early indicator of research impact. Journal of the Association for Information Science and Technology, 70(3), 271–282.
Xia, F., Su, X., Wang, W., Zhang, C., & Ning, Z. (2016). Bibliographic analysis of nature based on Twitter and Facebook altmetrics data. PLoS ONE, 11(12), e0165997.
Yu, H. (2017). Context of Altmetrics data matters: An investigation of count type and user category. Scientometrics, 111(1), 267–283.
Yu, H., Cao, X., Xiao, T., & Yang, Z. (2020). How accurate are policy document mentions? A first look at the role of altmetrics database. Scientometrics, 125(2), 1517–1540.
Yu, H., Xu, S., Xiao, T., Hemminger, B. M., & Yang, S. (2017). Global science discussed in local altmetrics: Weibo and its comparison with Twitter. Journal of Informetrics, 11(2), 466–482.
Zahedi, Z., & Costas, R. (2018). General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators. PLoS ONE, 13(5), e0197326. https://doi.org/10.1371/journal.pone.0197326
Zahedi, Z., Fenner, M., & Costas, R. (2014). Consistency among altmetrics data provider/aggregators: What are the challenges? Retrieved from https://openaccess.leidenuniv.nl/handle/1887/48265
Zhu, Y., & Procter, R. (2015). Use of blogs, Twitter and Facebook by UK PhD students for scholarly communication. Observatorio (obs*), 9(2), 29–46.
Acknowledgements
The research is supported by Humanity and Social Science Foundation of Ministry of Education of China (18YJC870023), National Natural Science Foundation of China (NO. 71804067), The Fundamental Research Funds for the Central Universities (No. 30920021203). The authors would like to thank Altmetric.com for providing access to the data.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yu, H., Murat, B., Li, L. et al. How accurate are Twitter and Facebook altmetrics data? A comparative content analysis. Scientometrics 126, 4437–4463 (2021). https://doi.org/10.1007/s11192-021-03954-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-021-03954-7