Review article
Interventions to support consumer evaluation of online health information credibility: A scoping review

https://doi.org/10.1016/j.ijmedinf.2020.104321Get rights and content

Highlights

  • Few interventions were designed based on theories and evaluated systematically.

  • Longitudinal effectiveness of the interventions is unclear.

  • Few interventions can assist consumers in real time while they search online.

Abstract

Purpose

Various interventions have been designed to help consumers better evaluate the credibility of online health information (OHI). However, assessing information credibility remained the most widely reported challenge by online health consumers. This review aims to provide an overview of major intervention approaches for improving consumer ability to evaluate OHI credibility in order to identify opportunities for future interventions.

Methods

A scoping review was performed. Seven relevant scientific databases were searched to identify articles that report the design and/or evaluation of interventions to support, facilitate, or assist consumers in assessing the credibility of OHI. Thirty-one articles met the inclusion criteria. Relevant content was extracted from the articles and all codes were validated by second coders.

Results

Three major intervention approaches for enhancing consumers' ability to evaluate OHI credibility were identified: educational program, algorithm, and interactive interface. The design of most interventions (particularly the credibility evaluation component) lacked the guidance of theories, and very few studies systematically evaluated their effectiveness in real online search contexts. Few interventions can provide spontaneous support to consumers while they search online.

Conclusion

Our understanding of what theoretical constructs contribute to effective OHI credibility evaluation interventions and how intervention outcomes should be measured remained limited. Future efforts need to focus on the design, development, test, and evaluation of theory-guided OHI credibility evaluation interventions that are scalable, sustainable, and can provide real-time support to consumers.

Introduction

The internet is the most popular information source for many consumers to obtain health information [[1], [2], [3]]. However, the quality of online health information (OHI) varies greatly, and the overall quality is low [4,5]. Low-quality information can undermine people’s ability to make informed decisions and lead to harmful consequences [6,7]. Therefore, the ability to evaluate the quality of OHI is considered a major component of eHealth literacy and is critical for consumers to make effective use of the internet as a source for health information [8].

Nevertheless, assessing information quality has been a significant challenge for online health consumers [9]. The challenge exacerbates when the complexity and heterogeneity of the internet information environment increases [10], particularly with the emergence of social media where anyone can publish health information and where low-quality and misinformation spreads quickly and widely [5,11]. Interventions are much needed to tackle this public health challenge.

Two systematic reviews [12,13] reviewed interventions that intend to enhance consumer eHealth literacy, of which evaluating OHI quality is a component. Some (but not all) studies included in these reviews contained an OHI quality evaluation component. However, specifics of the design and evaluation concerning the component were not the focus of the reviews. Therefore, they provided only a fragmented view of current efforts to enhance consumer OHI quality evaluation ability, gaps in the current efforts, and opportunities for improvement.

A more recent comprehensive review [14] focused on reviewing interventions that assist health consumers in autonomously finding reliable OHI related to chronic health conditions. Seven published studies and two gray literature reports were included in the review. Due to its limit to interventions targeting consumers who concerned about chronic conditions, they missed efforts targeting other user groups and online health consumers in general. Like the two systematic reviews, this review did not separate the OHI quality evaluation module from other modules (e.g., training on basic knowledge of stroke) in the interventions. Moreover, the review identified three types of interventions: interactive workshops, health literacy curriculum and community outreach, and online portals with support via videoconferencing; however, it did not include automatic methods (e.g., credibility-based ranking algorithms) that can enhance consumer ability to access more credible OHI. Such methods represent an important and also necessary approach to address the OHI quality evaluation challenge in the age of information overload.

This scoping review intends to address these significant gaps with two goals: (1) achieve a more comprehensive understanding of the current status of research on improving or augmenting consumers' ability to evaluate OHI credibility, and (2) generate ideas for future interventions. We focused on the following research questions: What intervention approaches have been designed to improve or augment consumer ability to evaluate OHI credibility? How were the interventions designed? How were they evaluated?

Based on the Merriam-Webster dictionary, we defined interventions broadly as acts designed to interfere with the course or the outcome of consumers’ evaluation of OHI credibility. We intended to be inclusive; thus, we did not limit ourselves to specific action mechanisms. The acts could be those targeting to actively build consumers’ OHI evaluation skills or those working by shaping, focusing, or nudging user attention in particular ways to augment their ability to evaluate OHI credibility. We defined credibility as consumers’ perception of the quality of OHI and the extent to which they are willing to trust the information [15,16]. This definition implies that, in this review, we view quality as objective features of OHI (e.g., accuracy and completeness), whereas credibility as subjective or perceived features of OHI (e.g., trustworthiness and usefulness). As most online health searchers do not equip with medical expertise to objectively assess OHI quality, it is appropriate to describe consumers’ evaluation of OHI quality as a subjective process (termed credibility evaluation) that is achieved by picking up and applying source, content, or design cues [15]. We broadly defined online health consumers as those who actively or passively access health and wellness information on the internet, irrespective of the mode of access (e.g., search, browse) and the motivation to access.

Section snippets

Methods

We adopted the scoping review method because our goal is to map the body of literature on the topic of supporting consumers in evaluating the credibility of OHI and a preliminary scanning of the literature suggests that the literature is diverse in research design and methodologies. Further, we aim to provide a descriptive overview of the literature to shed light on future intervention design, without critiquing the effectiveness of the existing interventions [17,18]. We followed the general

Basic characteristics of the included articles

Among the 31 articles included in the review, nine (29.0 %) were published in conferences, and 22 (71.0 %) were published in journals. The publication year ranged from 1999 to 2020, with 87.1 % (N = 27) of the articles being published after the year 2010 (Fig. 2). More than half of the studies (16, 51.6 %) were situated in general health contexts, and the remaining (15, 48.4 %) were situated in specific health conditions including cancer, HIV/AIDS, depression, diabetes, vaccine, skin disease,

Discussion

This scoping review identified three major intervention approaches to enhance consumers’ ability to evaluate OHI credibility: educational program, algorithm, and interactive interface. We discuss these approaches to shed light on potentially fruitful future research directions on this topic, followed by a discussion of the limitations of the current review.

Educational programs were in the form of face-to-face workshops or online portals and messages. A major problem of the included educational

Conclusion

This review identified three approaches to support consumers to evaluate OHI credibility: educational program, algorithm, and interface. Most of the included interventions lacked the guidance of theories in designing the OHI credibility evaluation component and lacked systematic real-life evaluation. Thus, our understanding of what theoretical constructs contribute to effective interventions and how intervention outcomes should be measured remained limited. Additionally, interventions that can

Author contributions

All authors have made a substantial, direct, intellectual contribution to this study.

Declaration of Competing Interest

The authors declare they have no conflict of interest for this study.

Summary points

What was already known on the topic?

  • Evaluating the credibility of online health information is a major challenge facing online health consumers.

  • Many interventions, such as checklists and third-party certifications, were created to help consumers overcome this challenge.

What this scoping review adds?

  • In addition to checklists and third-party certifications, three additional approaches to improve consumer ability to

Acknowledgments

We want to thank Lena Chu for her assistance with the article screening process and Dr. Andrew Dillon for constructive feedback to early drafts of the paper. The study is supported by the VPR Research & Creative Grant from the University of Texas at Austin.

References (65)

  • L. Wu

    Discovering differential features: adversarial learning for information credibility evaluation

    Inf. Sci.

    (2020)
  • M.A. Clarke

    Health information needs, sources, and barriers of primary care patients to achieve patient-centered care: a literature review

    Health Informatics J.

    (2016)
  • J.E. Volkman

    The National Cancer institute’s Health Information National Trends Survey [HINTS]: a national cross-sectional analysis of talking to your doctor and other healthcare providers for health information

    BMC Fam. Pract.

    (2014)
  • G. Eysenbach

    Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review

    JAMA

    (2002)
  • Y. Zhang et al.

    Quality of health information for consumers on the web: a systematic review of indicators, criteria, tools, and evaluation results

    J. Assoc. Inf. Sci. Technol.

    (2015)
  • C.D. Norman et al.

    eHEALS: the eHealth literacy scale

    J. Med. Internet Res.

    (2006)
  • S.Y. Rieh et al.

    Credibility: a multidisciplinary framework

    Annu. Rev. Inf. Sci. Technol.

    (2007)
  • L. Bode et al.

    See something, say something: correction of global health misinformation on social media

    Health Commun.

    (2018)
  • J. Car

    Interventions for enhancing consumers’ online health literacy

    Cochrane Database Syst. Rev.

    (2011)
  • I. Watkins et al.

    eHealth literacy interventions for older adults: a systematic review of the literature

    J. Med. Internet Res.

    (2014)
  • K. Lee

    Interventions to assist health consumers to find reliable online health information: a comprehensive review

    PLoS One

    (2014)
  • K.S. Freeman et al.

    An examination of factors that affect the credibility of online health information

    Tech. Commun.

    (2004)
  • J. Schwarz et al.

    Augmenting web pages and search results to help people find trustworthy information online

    Proceedings of the Annual SIGCHI Conference on Human Factors in Computing Systems

    (2011)
  • H. Arksey et al.

    Scoping studies: towards a methodological framework

    Int. J. Soc. Res. Methodol.

    (2005)
  • M.T. Pham

    A scoping review of scoping reviews: advancing the approach and enhancing the consistency

    Res. Synth. Methods

    (2014)
  • A.C. Tricco

    PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation

    Ann. Intern. Med.

    (2018)
  • D. Charnock

    DISCERN: an instrument for judging the quality of written consumer health information on treatment choices

    J. Epidemiol. Community Health

    (1999)
  • J.D. Stefan

    A search tool based on’ encapsulated’ MeSH thesaurus to retrieve quality health resources on the internet

    Med. Inform. Internet Med.

    (2001)
  • B. Horne

    Development and evaluation of a specifically designed website for haematopoietic stem cell transplant patients in Leeds

    Eur. J. Cancer Care

    (2016)
  • H.Y. Catherine

    Designing and evaluating a web-based self-management site for patients with type 2 diabetes-systematic website development and study protocol

    BMC Med. Inform. Decis. Mak.

    (2012)
  • S. Kalichman

    Closing the digital divide in HIV/AIDS care: development of a theory-based intervention to increase Internet access

    AIDS Care

    (2002)
  • D. Charnock et al.

    Learning to DISCERN online: applying an appraisal tool to health websites in a workshop setting

    Health Educ. Res.

    (2004)
  • Cited by (25)

    • Effects of information quality on information adoption on social media review platforms: moderating role of perceived risk

      2021, Data Science and Management
      Citation Excerpt :

      Perceived diagnosticity positively affects information adoption intention. Based on relevant literature, this study believes that information credibility is more concerned with consumers’ perception of the quality of social media online information and the degree of trust from the perspective of reputation (Song et al., 2021). With the development of Internet communication technology, almost anyone can post online comments on social media comment platforms.

    View all citing articles on Scopus
    View full text