Skip to main content
Log in

How accurate are Twitter and Facebook altmetrics data? A comparative content analysis

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Data accuracy is essential for reliable and valid altmetrics analysis. Although Twitter and Facebook altmetrics data are widely used for scholarly communication and scientific evaluation, few studies have tapped into their accuracy issue. Based on content analysis of random sample records over two phases, this study has investigated and compared the accuracy of Twitter and Facebook altmetrics data. Major conclusions are drawn as follows. (1) Three error types were identified from the altmetric data provider and six error types were identified from the altmetric data aggregator. Twitter and Facebook have shared most of the error types except for minor differences in the sub-categories. (2) The overall error rate is substantially high, being 17% and 32% for Twitter and Facebook respectively in April, 2019. However, except for publication date error and posting date error, the percentage of the other error types is relatively low (being around 3%). (3) The percentage of error types related to the dynamic nature of Twitter and Facebook is increasing over time, while percentage of error types concerning the bibliographic data is decreasing over time. (4) The error types are either “high seriousness low percentage” or “low seriousness high percentage”, therefore, they would probably not bring significant negative influence. (5) Underlying reasons of these error types are various. They could be attributable to the Twitter (or Facebook) user, Twitter (or Facebook) platform, altmetric database, as well as the third-party data provider. These results suggest that Twitter and Facebook altmetrics data in the Altmetric database are reliable on the whole, although there is still space for further improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Notes

  1. https://www.altmetric.com/explorer/highlights.

References

  • Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.

    Article  Google Scholar 

  • Bornmann, L., & Haunschild, R. (2018). Alternative article-level metrics: The use of alternative metrics in research evaluation. Embo Reports, 19(12), e47260.

    Article  Google Scholar 

  • Bornmann, L., Haunschild, R., & Adams, J. (2019). Do altmetrics assess societal impact in a comparable way to case studies? An empirical test of the convergent validity of altmetrics based on data from the UK research excellence framework (ref). Journal of Informetrics, 13(1), 325–340.

    Article  Google Scholar 

  • Duan, Q., & Pan, X. (2017). Identification of emerging topics in science using social media. Journal of the China Society for Scientific and Technical Information, 36(12), 1216–1223. (in Chinese).

    Google Scholar 

  • Enkhbayar, A., Haustein, S., Barata, G., & Alperin, J. P. (2020). How much research shared on facebook happens outside of public pages and groups? A comparison of public and private online activity around plos one papers. Quantitative Science Studies, 1(3), 1–22.

  • Fang, Z., & Costas, R. (2020). Studying the accumulation velocity of altmetric data tracked by altmetric.com. Scientometrics, 123, 1077–1101.

    Article  Google Scholar 

  • Fang, Z., Dudek, J., & Costas, R. (2020). The stability of Twitter metrics: A study on unavailable Twitter mentions of scientific publications. Journal of the Association for Information Science and Technology. https://doi.org/10.1002/asi.24344

    Article  Google Scholar 

  • Fraumann, G. (2017). Valuation of Altmetrics in Research Funding. (Unpublished master’s thesis). University of Tampere.

  • Haustein, S. (2016). Grand challenges in altmetrics: Heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423.

    Article  Google Scholar 

  • Hawkins, C. M., Duszak, R., & Rawson, J. V. (2014). Social media in radiology: Early trends in Twitter microblogging at radiology’s largest international meeting. Journal of the American College of Radiology, 11(4), 387–390.

    Article  Google Scholar 

  • Ministry of Education of China. (2020). The guidelines for improving the application of SCI related indicators and establishing proper evaluation direction in higher education institutions. Retrieved from http://www.moe.gov.cn/srcsite/A16/moe_784/202002/t20200223_423334.html

  • NISO. (2016). Outputs of the NISO alternative assessment project. Retrieved from https://www.niso.org/publications/rp-25-2016-altmetrics

  • Parvin, S. (2017). Facebook as a communication tool for an academic library: East West University Library, Bangladesh Perspective. The International Information & Library Review, 49(3), 237–247.

    Article  Google Scholar 

  • Thelwall, M., Tsou, A., Weingart, S., Holmberg, K., & Haustein, S. (2013). Tweeting links to academic articles. Cybermetrics, 17(1), 1–8.

    Google Scholar 

  • Weller, K., Bruns, A., Burgess, J., Mahrt, M., & Puschmann, C. (2014). Twitter and society. Peter Lang International Academic Publishers.

    Book  Google Scholar 

  • Wilkinson, S. E., Basto, M. Y., Perovic, G., Lawrentschuk, N., & Murphy, D. G. (2015). The social media revolution is changing the conference experience: Analytics and trends from eight international meetings. BJU International, 115(5), 839–846.

    Article  Google Scholar 

  • Wooldridge, J., & King, M. B. (2018). Altmetric scores: An early indicator of research impact. Journal of the Association for Information Science and Technology, 70(3), 271–282.

    Article  Google Scholar 

  • Xia, F., Su, X., Wang, W., Zhang, C., & Ning, Z. (2016). Bibliographic analysis of nature based on Twitter and Facebook altmetrics data. PLoS ONE, 11(12), e0165997.

    Article  Google Scholar 

  • Yu, H. (2017). Context of Altmetrics data matters: An investigation of count type and user category. Scientometrics, 111(1), 267–283.

    Article  Google Scholar 

  • Yu, H., Cao, X., Xiao, T., & Yang, Z. (2020). How accurate are policy document mentions? A first look at the role of altmetrics database. Scientometrics, 125(2), 1517–1540.

    Article  Google Scholar 

  • Yu, H., Xu, S., Xiao, T., Hemminger, B. M., & Yang, S. (2017). Global science discussed in local altmetrics: Weibo and its comparison with Twitter. Journal of Informetrics, 11(2), 466–482.

    Article  Google Scholar 

  • Zahedi, Z., & Costas, R. (2018). General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators. PLoS ONE, 13(5), e0197326. https://doi.org/10.1371/journal.pone.0197326

    Article  Google Scholar 

  • Zahedi, Z., Fenner, M., & Costas, R. (2014). Consistency among altmetrics data provider/aggregators: What are the challenges? Retrieved from https://openaccess.leidenuniv.nl/handle/1887/48265

  • Zhu, Y., & Procter, R. (2015). Use of blogs, Twitter and Facebook by UK PhD students for scholarly communication. Observatorio (obs*), 9(2), 29–46.

    Google Scholar 

Download references

Acknowledgements

The research is supported by Humanity and Social Science Foundation of Ministry of Education of China (18YJC870023), National Natural Science Foundation of China (NO. 71804067), The Fundamental Research Funds for the Central Universities (No. 30920021203). The authors would like to thank Altmetric.com for providing access to the data.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Houqiang Yu.

Appendix

Appendix

See Appendix Figs. 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Murat, B., Li, L. et al. How accurate are Twitter and Facebook altmetrics data? A comparative content analysis. Scientometrics 126, 4437–4463 (2021). https://doi.org/10.1007/s11192-021-03954-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-021-03954-7

Keywords

Navigation