Software Process Simulation Modeling: Systematic literature review

https://doi.org/10.1016/j.csi.2020.103425Get rights and content

Highlights

  • This paper describes a systematic review about Software Process Simulation Modelling (SPSM).

  • After the conducting phase, 36 primary studies were selected and studied in depth.

  • It is possible to conclude that SPSM continues to be a topic very addressed, but each contribution has particular goals.

  • Agent-Based Simulation and System Dynamics paradigms are increasing and decreasing in the last five years.

  • This paper concludes identifying some trends and future research directions on SPSM.

Abstract

Changes and continuous progress in logistics and productive systems make the realization of improvements in decision making necessary. Simulation is a good support tool for this type of decisions because it allows reproducing processes virtually to study their behavior, to analyze the impact of possible changes or to compare different design alternatives without the high cost of scale experiments. Although process simulation is usually focused on industrial processes, over the last two decades, new proposals have emerged to bring simulation techniques into software engineering. This paper describes a Systematic Literature Review (SLR) which returned 8070 papers (published from 2013 to 2019) by a systematic search in 4 digital libraries. After conducting this SLR, 36 Software Process Simulation Modeling (SPSM) works were selected as primary studies and were documented following a specific characterization scheme. This scheme allows characterizing each proposal according to the paradigm used and its technology base as well as its future line of work. Our purpose is to identify trends and directions for future research on SPSM after identifying and studying which proposals in this topic have been defined and the relationships and dependencies between these proposals in the last five years. After finishing this review, it is possible to conclude that SPSM continues to be a topic that is very much addressed by the scientific community, but each contribution has been proposed with particular goals. This review also concludes that Agent-Based Simulation and System Dynamics paradigm is increasing and decreasing, respectively, its trend among SPSM proposals in the last five years. Regarding Discrete-Event Simulation paradigm, it seems that it is strengthening its position among research community in recent years to design new approaches.

Introduction

In recent years, the application of process engineering principles to different environments as a basis for increasing the quality and excellence in organizations is worldwide accepted. In this sense, general reference standards [1], international project management guidelines [2,3] and reference standards in the Information and Communications Technology (ICT) context [4,5] promote that formal process management allows improving effectiveness and efficiency. This enhancement allows reducing costs, improving quality, and increasing the productivity and competitiveness of any organization.

This approach is well known as Business Process Management (BPM) [6], and its application can provide many advantages (e.g. faster time-to-market, higher productivity and efficiency, better product quality and reduced product cost, among others) [7]. BPM can be and has been successfully applied to different companies and software organizations are not an exception.

However, some difficulties in the software engineering context have been identified. For instance, Canfora compares Software Processes (SPs) with industrial processes [8]. This author identifies a set of features of SPs (e.g., SPs frequently evolve incorporating new lifecycles, SPs are complex and strongly influenced by unpredictable circumstances such as human work, SPs often need integration between frameworks and different technologies, etc.) and argue that BPM is usually applied by software companies to formally model their processes using formal notations. However, other aspects of BPM, such as the automation of SPs, are not usually approached because many software companies consider that these aspects include highly-complex tasks which could not be effectively automated with reduced costs [9]. In this context, the lack of automation of the process implies that each involved human role or work team perform the process execution manually and/or unilaterally [8]. This situation causes that monitoring, user-oriented execution, maintenance, and measurement of SPs could become difficult and expensive tasks.

Considering this complex scenario, it should be very interesting to check software process models and predict their behavior in different situations to reduce the risk of their deployments (into production environments) and subsequent costs as much as possible. Although different approaches exist to deal with complex processes [10], even pursuing multiple objectives beyond risk [11], using simulation techniques on processes is one of the well-known techniques to achieve that [12].

Simulation techniques help to understand and analyze processes for strategic management process improvement, forecasting or prediction, among others. The ultimate goal is being able of quantitatively estimate the impact of process design on its own performance. This research challenge has been traditionally addressed under the topic named Software Process Simulation Modeling (SPSM). This concept is originally introduced by Tarek Abdel-Hamid and Stuart Madnick who summarized their approach in their seminal book [13]. One of the most important factors to succeed in applying SPSM is to define a rigorous model that reliably represents the real system being modeled. For this reason, SPSM has been frequently perceived by companies as an “additional” cost to software projects slowing its wide adoption in the software business world. However, the number of publications related to this topic has been growing exponentially since [13], because simulation helps assess solutions to real situations in a virtual world, giving the opportunity to know the consequences within a risk-free environment.

As mentioned before, SPSM is a topic that has been studied by the community since 1980. In fact, over the last decades, multiple contributions have emerged in the research literature on SPSM and each of them has its own particular proposal. There are different reviews and surveys on SPSM [14], [15], [16], [17], [18], [19], [20], [21], [22], but it has not been possible to find current and formal reviews that provide a complete view on SPSM of the last five years. In addition, previous reviews do not categorize SPSM proposals under study according to their nature or Knowledge Area (KAs) within the field of Software Engineering (SE).

This paper aims to fill this gap by performing a systematic review to identify trends and directions for future research on SPSM after studying which proposals in this topic have been defined and the relationships and dependencies between these proposals in the last five years. Therefore, this Systematic Literature Reviews (SLR) contributes in four ways to ongoing research in the field of SPSM: (i) showing and reviewing the most widely common SPS paradigms that have been used from 2013 to 2019, as well as the methods or techniques that assist the SPSM; (ii) summarizing which purposes that have been addressed by SPSM; (iii) creating a paradigm-based taxonomy to classify software simulation proposals; and (iv) offering trends for future research. All these aspects will be addressed in this paper following meticulously Kitchenham's method [23], as it is described in the following sections.

Finally, this paper is structured as follows. Related works are briefly described in Section 2. Later, Section 3 and Section 4 describe the method used for the systematic review and its planning, respectively. Once decided how the SLR is going to be performed, the review protocol whose results are presented in Section 5 was conducted. Section 6 offer analysis and discussions on these results and Section 7 presents conclusions, open issues and future works.

Section snippets

Related works

Software process simulation is an active research topic over the last decades. This research topic was mentioned for the first time at 1998 in «International Workshop on Software Process Simulation Modeling» (ProSim workshop). These authors describe their conclusions of «why, what and how» software process has to be simulated in [20], where they also presented an overview of SPSM to identify the goals and scopes of simulation on software processes. Guidance in choosing an appropriate modeling

Review method

As mentioned above, we have followed Kitchenham's methodology [25] (which has been successfully applied in the software engineering context nuestra) to carry out our SLR. This methodology was updated over the years thanks to the fact that some authors have argued against it and/or presented improvements to Kitchenham's method. These aspects are mentioned below.

In [26], authors conclude the total number of systematic reviews has significantly increased over last decade. However, the Kitchenham's

Planning

Next sections describe in detail the planning process that has been carried out in this SLR. During this process, it has been identified the need for performing this review, the research questions have been formulated, and the review protocol has been defined and validated.

Conducting and quality results

Once the review protocol has been approved by all researchers, our review is carried out. For this purpose, firstly, the result of the selection process and data extraction are presented in Section 5.1. Also, a statistical study is performed. Later, Section 5.2 shows results after applying the quality questionnaire (Table 6) on each selected primary study.

Analysis, open issues and discussion

In this section, each research question of our systematic review is answered and discussed in detail to detect the weaknesses of each primary study.

Conclusion and future works

Defining software process models is a mechanism that allows abstract representations of procedures, methodologies, or set of steps in any topic of software engineering. However, the implementation of these process models in real environments can become costly and complex tasks due to characteristics of software processes [8]. In this context, it is very convenient to have mechanisms or techniques to verify and validate software process models before these ones are deployed into production

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research has been supported by the POLOLAS (TIN2016-76956-C3-2-R) and BadgePeople (TIN2016-76956-C3-2-R) projects of the Spanish Ministry of Economy and Competitiveness under ERD funds.

References (75)

  • Managing Successful Projects with Prince2

    (2009)
  • CMMI for Development, Version 1.3

    (2010)
  • Cartlidge, A., Hanna, A., Rudd, C., Macfarlane, I., Windebank, J., & Rance, S. (2007). An Introductory Overview of ITIL...
  • W.M.P. Van Der Aalst

    Business process management demystified: a tutorial on models, systems and standards for workflow management

    Advances in Petri Nets

    (2003)
  • J. Stark

    Product Lifecycle Management: 21st Century Paradigm for Product Realisation

    (2011)
  • G. Canfora

    Software process: characteristics, technology and environments

    UPGRADE

    (2004)
  • M.P. Papazoglou

    E-business: Organizational and Technical Foundations

    (2006)
  • J.-R. I. Barba et al.

    Time prediction on multi-perspective declarative business processes

    Knowl. Inf. Syst.

    (2018)
  • I.B. Jiménez-Ram'irez et al.

    Generating multi-objective optimized configurable business process models

  • J. Swanson

    Business dynamics—systems thinking and modeling for a complex world

    J. Oper. Res. Soc.

    (2002)
  • T.K. Abdel-Hamid et al.

    Software project dynamics: An Integrated Approach

    (1991)
  • H. Zhang et al.

    Reflections on 10 years of software process simulation modeling: a systematic review

  • Software process simulation over the past decade: trends discovery from a systematic review

    Integr. Vlsi. J.

    (2008)
  • H. Zhang et al.

    Software process simulation modeling: facts, trends and directions, in: neonatal

    Paediatr. Child Heal. Nurs.

    (2008)
  • H. Zhang et al.

    Software Process Simulation Modeling: An Extended Systematic Review

  • GaoC. et al.

    Software process simulation modeling: preliminary results from an updated systematic review

  • B. França et al.

    Are we prepared for simulation based studies in software engineering yet?

    CLEI Electronic Journal

    (2013)
  • M.I. Kellner et al.

    Software process simulation modeling: why? what? how?,

    J. Syst. Softw.

    (1999)
  • LiuD. et al.

    The role of software process simulation modeling in software risk management: a systematic review

  • N. Bin Ali et al.

    A systematic literature review on the industrial use of software process simulation

    J. Syst. Softw.

    (2014)
  • Monique Jansen-Vullers et al.

    Business process simulation–a tool survey

  • B. Kitchenham et al.

    Guidelines for performing systematic literature reviews in software engineering

    Engineering

    (2007)
  • H. Zhang et al.

    An empirical investigation of systematic reviews in software engineering

  • P. Bourque et al.

    SWEBOK v.3 – Guide to the Software Engineering – Body of Knowledge

    (2014)
  • S. Feyer et al.

    Integration of the Scientific Recommender System Mr. DLib Into the Reference Manager JABREF

    Integration of the Scientific Recommender System Mr. DLib Into the Reference Manager JABREF

    (2017)
  • D.J. Power

    Concepts and resources for managers

    Decis. Support Syst.

    (2002)
  • C.M. Schmucker et al.

    Systematic review finds that study data not published in full text articles have unclear impact on meta-analyses results in medical research

    PLoS ONE

    (2017)
  • View full text