Usability factors influencing the continuance intention of disaster apps: A mixed-methods study

https://doi.org/10.1016/j.ijdrr.2020.101874Get rights and content

Abstract

Mobile apps have the potential to aid disaster response by providing an avenue to distribute relevant and time-critical information to the public. Disaster apps already exist in the app markets. However, it is a challenge to engage users in retaining disaster apps on their smartphones. A mixed-methods approach is used in this study to investigate whether usability factors affect users' intention to continue to use an app (referred to here as continuance intention). First, quantitative methods, applying structural equation modelling with survey data from 271 disaster app users, tested a usability–continuance model. Second, a qualitative usability inquiry, using in-depth interviews with 18 participants, explored the users' insights of the relationships identified from the quantitative modelling. The results showed five usability factors to have significant influence on continuance intention. The key positive influencers are (1) users’ perceptions as to whether the app delivers its function (app utility); (2) whether it does so dependably (app dependability); and (3) whether it presents information that can be easily understood (user-interface output). Subsequently, too much focus on (4) user-interface graphics and (5) user-interface input can discourage continuance intention. The insights from the qualitative inquiry provide further meaning and context to these relationships. The results have practical implications for designers and developers, guiding what factors to focus on to enhance the continuance intention of disaster apps.

Introduction

The large-scale use of new technologies, such as mobile application (apps), by ordinary people, has improved communications with the public during disasters [1]. Mobile apps intended for managing disasters (hereinafter referred to as disaster apps) can aid the public in retrieving, understanding, and using time- and location-critical information to enhance their decision-making processes during a disaster situation [2]. Disaster apps are becoming popular among the general public [3]. Moreover, the public tends to perceive disaster apps to be more reliable than social media as a communication platform between citizens and authorities [4]. Crisis management authorities also have put effort into developing and promoting official disaster apps. For example, the Red Cross and Red Crescent Societies have released or are releasing disaster apps in different countries [5]. While disaster apps are becoming more readily available, only a limited number of scientific publications have looked into understanding the users and their perceptions on the usability of these disaster apps [6]. This study investigates disaster apps and in particular looks into apps that are one-way communication tools available to the public that provide credible information about natural hazards, such as severe weather, wildfires, earthquakes, or multi-hazards.

Disaster apps, like most publicly targeted apps, are accessible through app markets like Google Play [7] and the iOS App Store [8]. However, despite the existence of numerous apps in the market, one out of four apps is abandoned after the initial download and use [9,10]. Especially when users may have alternative options, and they can quickly uninstall apps after the initial experience and interaction with the app [11,12]. In other words, apps have a challenge of maintaining users' continuance intention, a term which refers to users' intention to continue using a particular technology after its initial acceptance [13]. The impressions made during early usage after download become critical to users’ intention to retain or abandon apps [14].

Usability and its relationship to continuance intention are important for disaster apps. Disaster apps are not necessarily designed for everyday use [15] and users do not expect to interact with disaster apps as frequently as other types of apps [16]. Users will rely on the usability of the technologies to facilitate critical decisions during disaster situations [17]. Disaster apps will be of little or no value if users find them unusable when interacting with them or abandon them after the initial download and interaction [4,18]. Some research has been published on usability and continuance intention for mobile apps, but none, to the authors' knowledge, has addressed these in the context of disaster apps. There is a need for more in-depth research into citizens' perceptions towards disaster apps and the effect of usability to users’ continuance intention. The paper seeks to answer the question: How does usability affect continuance intention of disaster apps? This paper intends to fill the gap in research by investigating whether usability factors affect continuance intention, in the context of disaster apps that are meant for use by the general public.

Section snippets

Apps in critical contexts

In crisis situations, well-designed technological systems have helped responders and agencies to retrieve information, aiding them to make better decisions on the potential threats and response options [19,20]. Safety-critical systems literature has acknowledged the importance of usability, as the lack of usability of technical systems can lead to users’ compromised safety [21]. Most usability studies in the disaster and crisis management field, however, have focussed on tools for responders,

Theory and hypotheses

Hoehle and Venkatesh's [31] usability-continuance model fits the aim of the study in investigating usability factors that can improve the likelihood that people will continue using apps. However, this study also recognises the need to investigate usability and continuance intention in the context of disaster apps. Tan et al. [2] had investigated qualitatively whether Hoehle and Venkatesh's [31] mobile app usability model adequately represents the context for disaster apps. In addition to the

Methodology

A sequential explanatory mixed-method research approach, based on Creswell and Plano Clark [65]; was chosen for this study. The study first focusses on the quantitative component to test the conceptual model and determine the usability factors affecting continuance intention. It then uses qualitative methods to assist in understanding and explaining the quantitative results. This mixed-methods study received peer-reviewed approval under the Massey University code of ethical conduct for

Quantitative results

Participants had a median age of 39.5 (see Fig. 2 for survey participants’ age range). Fifty-four per cent of the respondents identified as female, 45% as males, and 1% identified as gender diverse or chose not to disclose their gender. Two-hundred and eight of the participants revealed their ethnicity (63 chose not to disclose). Of those who disclosed their identity, 76% identified as New Zealand European, 4% identified as Māori, 1% identified as Chinese, and 21% identified with other

Qualitative results

To further the relationships of the seven factors to continuance intention, this study also engaged with the users to understand their perspectives. A usability inquiry using a prototype was conducted with 18 members of the public. The inquiry participants were all experienced smartphone users. The participants had a median age of 29.5 years and a range between 18 to 60 years old. Twelve of the participants identified as female and the remaining six as male. At the time of the interview, all

Discussion

This study pioneers usability–continuance research on disaster apps. It is one of the first to start the discourse for future research on the usability of disaster apps as well as on the users' intentions to use publicly available technologies for disaster management. While much research work has explored the potential functionalities of apps for disasters, the usability of these apps is often overlooked [6]. Furthermore, most continuance intention studies have looked at usability in an

Conclusion

This study has highlighted that disaster apps’ usability factors have distinct relationships with continuance intention that may be different from other generic apps. For example, UI graphics has a negative relationship with continuance intention. The results provide an inference that disaster app users can negatively perceive an app to have too many visual elements. Further inquiry revealed that a fussy app could leave an impression that the interface will be too confusing and complicated for

Funding sources

This paper is part of a PhD project that received financial support from Education New Zealand and Massey University College of Humanities and Social Sciences.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

The full version of this manuscript has not been published elsewhere. We disclose that the initial results of the quantitative component of this mixed-methods study were presented as a work-in-progress paper at the 1st Asia Pacific Conference on Information Systems in Crisis Response and Management, 5–7 November 2018,

References (89)

  • H. Treiblmaier et al.

    Exploratory factor analysis revisited: how robust methods support the detection of hidden multivariate data structures in IS research

    Inf. Manag.

    (2010)
  • E. Kline et al.

    Convergent and discriminant validity of attenuated psychosis screening tools

    Schizophr. Res.

    (2012)
  • C. Xu et al.

    A customer value, satisfaction, and loyalty perspective of mobile application recommendations

    Decis. Support Syst.

    (2015)
  • C. Rieger et al.

    Towards the definitive evaluation framework for cross-platform app development approaches

    J. Syst. Software

    (2019)
  • G.L. Foresti et al.

    Situational awareness in smart environments: socio-mobile and sensor data fusion for emergency response to disasters

    J. Ambient Intell. Humanized Comput.

    (2015)
  • M.L. Tan et al.

    Modified usability framework for disaster apps: a qualitative thematic analysis of user reviews

    Int. J. Disaster Risk Sci.

    (2020)
  • D.J. Bachmann et al.

    Emergency preparedness and disaster response: there's an app for that

    Prehospital Disaster Med.

    (2015)
  • S. Appleby-Arnold et al.

    Truth, trust, and civic duty: cultural factors in citizens' perceptions of mobile phone apps and social media in disasters

    J. Contingencies Crisis Manag.

    (2019)
  • Mobile Apps

    (2020)
  • Google. (n.d.). Google Play. Retrieved August 9, 2016, from...
  • Apple Inc. (n.d.). The app store. Retrieved April 7, 2020, from...
  • Deloitte

    So many apps so little download

  • Localytics

    24% of Users Abandon an App after One Use

    (2017)
  • A. Kim et al.

    User experience and the multi-stage adoption of mobile apps

    J. Inf. Technol. Appl. Manag.

    (2014)
  • A. Bhattacherjee

    Understanding information systems continuance: an expectation-confirmation model

    MIS Q.

    (2001)
  • Y. Bang et al.

    Predicting post-adoption usage of information technology : a large-scale data analysis of mobile app download and usage behavior

  • C. Reuter et al.

    Social media in emergencies: a representative study on citizens' perception in Germany

    Proc. ACM Hum. Comput. Interact.

    (2017)
  • P. Sarshar et al.

    On the usability of smartphone apps in emergencies: an HCI analysis of GDACSmobile and SmartRescue apps

  • E. Bopp et al.

    Sorting the good from the bad smartphone applications to alert residents facing disasters – experiments in France

  • L. Yang et al.

    On-site information systems design for emergency first responders

    J. Inf. Technol. Theor. Appl.

    (2009)
  • S.T. Kwee-Meier et al.

    Integrated information visualisation and usability of user interfaces for safety-critical contexts

  • J. Kuula et al.

    Smartphones as an alerting, command and control system for the preparedness groups and civilians: results of preliminary tests with the Finnish police

  • R. Prasanna et al.

    Information systems architecture for fire emergency response

    J. Enterprise Inf. Manag.

    (2017)
  • E. Elmasllari et al.

    Learning from non-acceptance: design dimensions for user acceptance of e-triage systems

  • D. Gómez et al.

    A review on mobile applications for citizen emergency management

  • C. Reuter et al.

    Katwarn, NINA, or FEMA? Multi-method study on distribution, use, and public views on crisis apps

  • B. Göransson et al.

    The usability design process: integrating user-centered systems design in the software development process

    Software Process Improv. Pract.

    (2003)
  • Usability

  • N. McNamara et al.

    Functionality, usability, and user experience

    Interactions

    (2007)
  • R. Harrison et al.

    Usability of mobile applications: literature review and rationale for a new usability model

    J. Interact. Sci.

    (2013)
  • H. Hoehle et al.

    Mobile application usability: conceptualisation and instrument development

    MIS Q.

    (2015)
  • D. Zhang et al.

    User-centered context-aware mobile applications ― the next generation of personal mobile computing

    Commun. Assoc. Inf. Syst.

    (2009)
  • A.I. Wasserman

    Software engineering issues for mobile application development

  • T. Mentler

    Applying usability engineering to interactive systems for crisis and disaster management

  • Cited by (18)

    • Standing up for or against: A text-mining study on the recommendation of mobile payment apps

      2021, Journal of Retailing and Consumer Services
      Citation Excerpt :

      As such, by ensuring that a mobile payment system works well as expected, users will likely perceive it to be useful and this could explain why they would recommend the mobile payment system. Likewise, usability has been widely acknowledged (e.g. Baek and Yoo, 2018; Kokini et al., 2012; Ling and Salvendy, 2013; Mack and Sharples, 2009; Tan et al., 2020) as a key factor that determines the success of a product in many ways (e.g. continuance use, loyalty, customer engagement, competitive advantage, user performance etc.). Consequently, having poor usability can have dire consequences for the product owner.

    • Why should I use a multi-hazard app? Assessing the public's information needs and app feature preferences in a participatory process

      2021, International Journal of Disaster Risk Reduction
      Citation Excerpt :

      Which features does the public wish for a multi-hazard app? Recent studies have mainly assessed user needs through interviews, app store comments, surveys [31,32] and by letting participants/users test pre-developed prototypes [3,53,54]. Thereby, researchers or developers gain insights about users' response to already existing apps or prototypes.

    • Influences on the decision to implement electronic health records in Indonesia

      2023, Theoretical and Conceptual Frameworks in ICT Research
    View all citing articles on Scopus
    View full text