An ontological analysis of software system anomalies and their associated risks

Abstract

Software systems have an increasing value in our lives, as our society relies on them for the numerous services they provide. However, as our need for larger and more complex software systems grows, the risks involved in their operation also grows, with possible consequences in terms of significant material and social losses. The rational management of software defects and possible failures is a fundamental requirement for a mature software industry. Standards, professional guides and capability models directly emphasize how important it is for an organization to know and to have a well-established history of failures, errors and defects as they occur in software activities. The problem is that each of these reference models employs its own vocabulary to deal with these phenomena, which can lead to a deficiency in the understanding of these notions by software engineers, causing potential interoperability problems between supporting tools, and, consequently, a poorer adoption of these standards and tools in practice. In this paper, we address this problem of the lack of a consensual conceptualization in this area by proposing two reference conceptual models: an Ontology of Software Defects, Errors and Failures (OSDEF), which takes into account an ecosystem of software artifacts, and a Reference Ontology of Software Systems (ROSS), which characterizes software systems and related artifacts at different levels of abstraction. Moreover, we use OSDEF and ROSS to perform an ontological analysis of the impact of defects, errors and failures of software systems from a risk analysis perspective. To do that, we employee an existing core ontology, namely, the Common Ontology of Value and Risk (COVR). The ontologies presented here are grounded on the Unified Foundational Ontology (UFO) and based on well-known and widely-accepted standards, professional and scientific guides and capability models. We demonstrate how this approach can suitably promote conceptual clarification and terminological harmonization in this area.

Introduction

Software plays an essential role in modern society and it has become indispensable in many contexts of our lives, such as social, business, and personal contexts. This essential role motivates a number of research initiatives aimed at understanding the nature of software, and its relation to us. A shared conception in those initiatives is that software is a complex (social) artifact [1], [2], [3]. This notion comes from the fact that a modern software system can be understood as the combination of a series of interacting elements, specifically organized to provide a set of functionalities to fulfill particular human purposes [4], [5]. Moreover, software is constantly growing, not only in simple measures such as the number of lines of code, but also according to other factors, like complexity, criticality and degree of heterogeneity [6]. This makes it harder and more costly to maintain and evolve software, and this may be the starting point for many problems in the software life-cycle.

Besides its importance in our society, software is special also because it is capable of existing through time, being replicated millions of times and having dozens of different versions while still maintaining its identity [2]. A classic example of these intrinsic properties can be observed in Microsoft Windows, an operating system that has been created over 30 years ago, received many updates and was released under many different versions, but still maintains its identity as Microsoft’s operating system.

Despite their special properties, software systems are still artifacts, susceptible to failures, defects and faults that can range from having a small impact to being critical, thus, potentially causing significant material and social losses. Concepts such as problem, anomaly, bug and glitch are usually treated indistinctly, while potentially having different ontological semantics. This informal use, as common and practical as it may be in our daily conversations, can be the source of ambiguity and false-agreement problems, since the concept anomaly is frequently overloaded, thus, referring to entities with distinct ontological natures. In a more formal environment, this construct overload may lead to communication problems and losses. Because of that, and as defended in scientific literature, international standards and maturity models, it is important to have a precise way of classifying different types of software anomalies.

For example, the Guide to Software Engineering Body of Knowledge (SWEBoK) [5] emphasizes the need of a consensus about anomaly characterization, and discusses how a well-founded classification could be used in audits and product reviews. Moreover, the CMMI [7] model advocates that organizations should create or reuse some form of classification method for defects and failures. It also suggests the use of a defect density index for many work products that are part of the software development process.

A proper classification scheme can enable the development of different types of anomaly profiles that can be produced as an indicator of product quality. Also, systematically classifying software anomalies that may occur at design-time or runtime is a rich source of data that can be used to improve processes and avoid the occurrence of anomalies in future projects [8]. Finally, defects, faults and failures have a negative impact on important aspects of software, such as reliability, efficiency, overall cost and, ultimately, lifespan. Hence, a better understanding of the ontological nature of these concepts and how they relate to other software artifacts (e.g. requirements, change requests, reports and tests cases) can improve the way an organization deals with these issues, ultimately reducing costs with activities such as configuration management and software maintenance.

Although there are some proposals for classifying different terms for software anomalies, there is no reference model or theory that elaborates on the nature of different software anomalies. In other words, to the best of our knowledge, there is no proper reference ontology [9] focused on representing software defects, errors and failures. In order to address this gap, we propose a reference Ontology of Software Defects, Errors and Failures (OSDEF). This ontology takes into account different types of anomalies that may exist in software-related artifacts and that are recurrently mentioned in the set of the most relevant standards in the area. Furthermore, we recognize the importance of analyzing such anomalies in terms of the risk their presence ensues to software systems and, in particular, to these systems as bearers of value to some agent. In order to do that, we needed to elaborate on the relation between software systems and other software artifacts at different levels of abstraction. For this, we developed a Reference Ontology on Software Systems (ROSS). OSDEF and ROSS are then analyzed from this risk analysis perspective by leveraging on the Common Ontology of Value and Risk (COVR) [10].

OSDEF and ROSS are developed following the process defined by the Systematic Approach for Building Ontologies (SABiO) [11] and grounded on the Unified Foundational Ontology (UFO) [12], [13], including UFO’s Ontology of Events (UFO-B) [14], [15]. In order to elicit consensual information about the domain, we analyze relevant standards, guides and capability models such as CMMI [7], SWEBoK [5], IEEE Standard Classification for Software Anomalies [8], IEEE Standard for System, Software, and Hardware Verification and Validation [16], as well as complementary current Software Engineering literature. Finally, the ontologies are evaluated by verification and validation techniques recommended by SABiO.

This paper is an extended version of [17]. In this version, we present as original contributions: an extension of the original OSDEF ontology to incorporate the notion of run-time vulnerabilities, which inhere in loaded program copies as opposed to programs; the Reference Ontology on Software Systems (ROSS); an ontological analysis of three famous cases of software failures. The latter is done by instantiating them with the concepts from OSDEF, ROSS and the risk analysis perspective provided by COVR.

The remainder of this paper is structured as follows. Section 2 briefly presents the foundations used for developing the ontologies proposed in this work. In that section, we briefly introduce the reader to: the SABiO method, the foundational ontology UFO, and three core ontologies, namely, COVR, but also the Software Process Ontology (SPO) [18] and the Software Ontology (SwO) [19]). The last two ontologies have been reused to create OSDEF and ROSS. OSDEF and ROSS are presented in Sections 3 An ontology of software defects, errors and failures, 4 A reference ontology of software systems , respectively. Section 5 evaluates the proposed ontologies That section also presents the instantiation of ROSS and OSDEF from a risk analysis perspective by reusing them in combination with COVR. Section 6 discusses related work. Finally, Section 7 concludes the paper by presenting some final considerations.