Management of quality requirements in agile and rapid software development: A systematic mapping study

https://doi.org/10.1016/j.infsof.2019.106225Get rights and content

Abstract

Context

Quality requirements (QRs) describe the desired quality of software, and they play an important role in the success of software projects. In agile software development (ASD), QRs are often ill-defined and not well addressed due to the focus on quickly delivering functionality. Rapid software development (RSD) approaches (e.g., continuous delivery and continuous deployment), which shorten delivery times, are more prone to neglect QRs. Despite the significance of QRs in both ASD and RSD, there is limited synthesized knowledge on their management in those approaches.

Objective

This study aims to synthesize state-of-the-art knowledge about QR management in ASD and RSD, focusing on three aspects: bibliometric, strategies, and challenges.

Research method

Using a systematic mapping study with a snowballing search strategy, we identified and structured the literature on QR management in ASD and RSD.

Results

We found 156 primary studies: 106 are empirical studies, 16 are experience reports, and 34 are theoretical studies. Security and performance were the most commonly reported QR types. We identified various QR management strategies: 74 practices, 43 methods, 13 models, 12 frameworks, 11 advices, 10 tools, and 7 guidelines. Additionally, we identified 18 categories and 4 non-recurring challenges of managing QRs. The limited ability of ASD to handle QRs, time constraints due to short iteration cycles, limitations regarding the testing of QRs and neglect of QRs were the top categories of challenges.

Conclusion

Management of QRs is significant in ASD and is becoming important in RSD. This study identified research gaps, such as the need for more tools and guidelines, lightweight QR management strategies that fit short iteration cycles, investigations of the link between QRs challenges and technical debt, and extension of empirical validation of existing strategies to a wider context. It also synthesizes QR management strategies and challenges, which may be useful for practitioners.

Introduction

In the current era, which is characterized by continuous technological breakthroughs, high quality has become imperative for the acceptance of software systems. Non-secure business applications, unreliable Internet of Things platforms, or inefficient 5G-based systems can no longer survive in the context of pervasive IT. Evidence supports these claims; market prospects indicate that up to 26% of firms’ IT budgets are dedicated to software quality assurance and testing, and they predict an increase to 33% in the next three years [1]. In addition, some studies have shown the impact of overlooking quality-related aspects on the costs of software development and maintenance [2], [3], [4]. For instance, Ramesh et al. [3] reported that overlooking quality-related aspects in the early stages of development resulted in degradation of system quality and required redevelopment of the whole system.

However, software quality is elusive [5]; it is not easy to define or measure. In this context, quality requirements (QRs) play a crucial role in dealing with quality. QRs are the desired qualities of a system to be developed, such as maintainability, reliability, availability, usability, and integrity [6]. Although QRs have some similarities to their functional counterpart—namely, functional requirements—they are unique in other respects, including their meaning, how they are expressed, and how they are measured. While these challenges exist regardless of the approach used to develop software, they are particularly prominent in agile software development (ASD) which entail incremental and iterative software development methods guided by agile manifesto [7], and rapid software development (RSD) .1 ASD methods (e.g. Scrum, and XP), and RSD approaches (e.g. continuous deployment, continuous delivery, and DevOps, which extend ASD’ capability by shortening the time to delivery of software), are widely adopted throughout the industry because they place focus on continuous delivery of valuable software and customer satisfaction [8,9]. In such approaches, functional requirements tend to be favored over QRs [8,10,11], leading QRs to be improperly documented [12]. Consequently, quality aspects, such as system security, performance, and usability, are often compromised [3].

Neglecting QRs introduces bottlenecks in ASD [13] and RSD processes. Specifically, Bellomo et al. [14] reveal how neglecting QRs in early phases of development may compromise performance and stability. Additionally, inadequate or missing knowledge of QRs and functional requirements often incur technical debt in ASD, leading to the need for rework, and increased maintenance costs [4]. Technical debt due to QRs results in quality issues that become difficult to test in later phases [15]. In this regard, systematic management of QRs is important.

Despite the need to better understand the management of QRs in ASD and RSD, the body of knowledge on this topic is not well structured. While there are secondary studies [10,11,[16], [17], [18], [19], [20], [21], [22]] that examine state-of-the-art of requirements engineering (RE) in the context of ASD, only Alsaqaf et al. [10] and Villamizar et al. [22] specifically focused on QRs. However, Alsaqaf et al. [10] did so only in the context of large-scale ASD, whereas Villamizar et al. [22] examined only how security is handled in ASD. Existing secondary studies of RSD [8,[23], [24], [25]] also do not specifically focus on QRs. This makes it difficult for researchers and practitioners to obtain a clear understanding of QRs and their management in ASD and RSD. Uncovering important QRs in such contexts, related challenges and gaps would be beneficial. In this paper, we aim to fill this research gap by systematically identifying, structuring, analyzing, and assessing the quality of the scientific literature on management of QRs in ASD and RSD. In short, our systematic mapping study (SMS) has the following aims:

  • 1.

    To identify and classify the existing scientific research on management of QRs in ASD and RSD.

  • 2.

    To assess the quality of existing empirical work on the topic in terms of research rigor and industrial relevance.

  • 3.

    To identify and classify QR management strategies in ASD and RSD.

  • 4.

    To identify and classify the challenges of managing QRs in ASD and RSD.

Our work contributes to the existing body of knowledge on RE in ASD and RSD. For academia, we identify important research gaps that need further investigation, such as tools and guidelines for managing QRs, lightweight QR management strategies that can be used with short iteration cycles, the link between QRs and technical debt, and empirical validation of existing strategies in a wider context. Practitioners can utilize the findings of this study to improve their understanding of QR management in ASD and RSD. Consequently, they may address and eventually overcome the quality-related issues that arise in ASD and RSD. In such scenarios, it is important they assess the relevance and applicability of the management strategies in their development contexts. The rest of the paper is organized as follows. Section 2 provides a background and related work on the topic. Section 3 presents the research methodology followed to conduct the SMS. Section 4 presents the findings of the research. Section 5 discusses the implications of the findings. Finally, Section 6 summarizes and concludes the study.

Section snippets

Background and related work

This section presents some background information about QRs and their management in ASD and RSD. Moreover, it presents secondary studies on agile requirement engineering and RSD.

Research methodology

We performed an SMS following the guidelines proposed by Kitchenham et al. [51] and Peterson et al. [52]. We also adopted the snowballing guidelines proposed by Wohlin [53] to systematically search primary studies. SMS allows one to structure the evidence regarding a domain in which there is limited evidence or a broad topic [51]. Thus, in our study, SMS enabled us to structure the broad research on QR management in ASD. It was also suitable for determining and structuring the limited evidence

Results and analysis

In this section, we present the results and discuss how they relate to our research questions. Section 4.1 provides an overview of the research on the topic. Section 4.2 presents the findings regarding QR management strategies, and Section 4.3 presents the findings regarding the challenges of managing QRs. Finally, Section 4.4 maps the QR management strategies with the challenges.

Discussion

In this section, we discuss the implications of our findings for SE research, SE industry, and SE education, and compare our findings with related work.

Conclusion

Due to advancements in technology and highly competitive markets, ASD and RSD are a necessity for fast delivery of software. Ensuring high software quality is greatly important in such development approaches. QRs affect software quality and have economic implications on software development, and they play an important role in determining the success of ASD and RSD projects.

In this study, we explored the management of QRs in ASD and RSD through classification and synthesis of 156 primary

Acknowledgments

This work is the result of the Q-Rapids project, which has received funding from the European Union's Horizon 2020 research and innovation program under grant agreement no. 732253.

References (68)

  • Capgemini, Sogeti, M. Focus, World Quality Report 2018-19, 2018....
  • C. Jones et al.

    The economics of software quality

    Add. Wesl. Profess.

    (2011)
  • B. Ramesh et al.

    Agile requirements engineering practices and challenges: an empirical study

    Inf. Syst. J.

    (2010)
  • B. Kitchenham et al.

    Software quality: the elusive target

    IEEE Softw.

    (1996)
  • K. Wiegers et al.

    Software Requirements

    (2013)
  • K. Beck et al.
    (2001)
  • P. Rodríguez et al.

    Survey on agile and lean usage in finnish software industry

    Int. Symp. Empir. Softw. Eng. Meas.

    (2012)
  • W. Alsaqaf et al.

    Quality requirements in large-scale distributed agile projects – A Systematic literature review

  • S. Bartsch

    Practitioners’ perspectives on security in agile development

  • L. Cao et al.

    Agile requirements engineering practices: an empirical study

    IEEE Softw.

    (2008)
  • S. Bellomo et al.

    A study of enabling factors for rapid fielding combined practices to balance speed and stability

  • A. Martini et al.

    The danger of architectural technical debt: contagious debt and vicious circles

  • V.T. Heikkilä et al.

    A mapping study on requirements engineering in agile software development

  • D.A. Magües et al.

    Requirements engineering related usability techniques adopted in agile development processes

  • P. Heck et al.

    A systematic literature review on quality criteria for agile requirements specifications

    Softw. Qual. J.

    (2016)
  • K. Elghariani et al.

    Review on agile requirements engineering challenges

  • H. Villamizar et al.

    A systematic mapping study on security in agile requirements engineering

  • M.V. Mäntylä et al.

    On rapid releases and software testing: a case study and a semi-systematic literature review

    Empir. Softw. Eng.

    (2015)
  • C.R. Klaus Pohl

    Requirements engineering fundamentals a study guide for the certified professional for requirements engineering exam

    Rocky Nook

    (2015)
  • L. Chung et al.

    On non-functional requirements in software engineering

    Concept. Model. Found.

    (2009)
  • J. Doerr et al.

    Non-functional requirements in industry - three case studies adopting an experience-based NFR method,

  • J. Mylopoulos et al.

    Representing and using nonfunctional requirements: a process-oriented approach

    IEEE Trans. Softw. Eng.

    (1992)
  • M. Shaw

    Truth vs knowledge: the difference between what a component does and what we know it does

  • R.L. Glass
    (1998)
  • Cited by (0)

    View full text