The prevalence and severity of persistent ambiguity in software requirements specifications: Is a special effort needed to find them?☆
Introduction
The primary goal of requirements engineering (RE)1 is to capture in a requirements specification (RS) all of the requirements that the client, users, and other stakeholders believe to be imperative in the computer-based system (CBS) being developed [2]. Ambiguity in an RS, by which the RS or a piece of it could have multiple meanings, could cause programmers to implement the CBS incorrectly from the client's viewpoint, resulting in major code rewrites, leading to delays in delivery, and introducing even more defects [3]. Therefore, it is worth finding ways to avoid, to even prevent, and failing those, to detect ambiguity in RSs [4], [5], [6], [7], [8], [9].
Project failure has often been attributed to ambiguities in RSs. For example, Gause lists too much unrecognized disambiguation in RSs as one of the five most important sources of requirements failure [10]. This attribution claim has not been conclusively empirically proved.
The underlying claim of the research on ambiguity and on ways to avoid, prevent, and detect ambiguities in RSs is that
- –
only by detecting each ambiguous statement in an RS for a CBS, finding each possible interpretation of the statement and rewriting the statement so that its only interpretation is what the CBS's stakeholders intend,
- –
can it be assured that what is implemented is what is intended.
This paper describes case study research that tried its very best to come to conclusions different from those of de Bruijn and Philippo. The attempt failed, and the study ended up confirming their conclusions. In the rest of this paper, Section 2 discusses the role of ambiguity in RE and briefly describes methods and tools for avoiding, preventing, and detecting ambiguity in RSs. Section 3 lays out the general context and the vocabulary assumed in the rest of the paper. Section 4 gives a detailed description of past attempts to validate the importance of ambiguity detection during RE. Section 5 describes the objectives and the approach taken by our research that attempted to falsify de Bruijn's and Philippo's conclusions. Section 6 defines and gives examples of what this paper calls persistent ambiguities, those that stakeholders do not know about and therefore are likely not to be discussed during RE. Section 7 lays out the research plan for the case studies. Section 8 briefly describes the conduct of the case studies, carried out according to the plan. Section 9 reports the data and the results of the case studies, and Section 10 comes to the verdict about the attempt to falsify de Bruijn's and Philippo's results. Section 11 discusses the results and their implications on the practice of RE. Section 12 concludes the paper with a discussion of future work.
Section snippets
Background on ambiguity in RE
Almost all RSs are written in natural language (NL) [13]. Even when an RS is written in a formal language or with UML diagrams, the RE process still begins with ideas expressed in NL [9]. NLs are inherently ambiguous. Therefore, since NL is unavoidable in the RE process, there is no escaping ambiguous RSs [6].
Gervasi and Zowghi point out that ambiguity in NL RSs “has long been pictured as one of the worst enemies of the specifier” [14]. Consequently, as mentioned, there has been lots of
Context and vocabulary
This section establishes some context about CBS developments in which the phenomenon of ambiguity in RSs plays a role.
An ambiguous statement in the RS for a CBS is a statement that can be interpreted in multiple ways, thus giving rise to the possibility that different people will implement the statement in different ways. The phenomenon of ambiguity in RSs has been described thoroughly elsewhere [7], [8], [40]. In the interest of saving space in this paper, the reader desiring a more detailed
Past attempts to validate importance of ambiguity detection in RE
Fabian de Bruijn, under supervision by H. Dekker, tried to determine the effect of ambiguity in a project's RS on the project's downstream success or failure [11]. He analyzed the 279-requirement RS for one failed CBS development project. With his analysis, he attempted to answer the following research questions [11]:
- 1.
How many requirement statements are ambiguous?
- 2.
How many problems were caused by ambiguous requirements?
De Bruijn's analysis determined that only one not very severe defect in the
Research objectives and approach
We decided to try our damndest to falsify de Bruijn's and Philippo's result, using the RSs from three different projects3 that were well into deployment, using a totally different sampling method designed to both (1) reduce the number of ambiguities to investigate to a manageable number and (2) increase the chances for falsification. We used a purposive sampling
Persistent ambiguities
Language usage is constantly evolving. Occasionally people's usage does not follow the rules of language for brevity or emphasis. Some sentences, although grammatically problematic, have become common usage. The description below of ambiguities in RSs focuses on how people use language, grammatically correct or not, and how that usage might cause ambiguity. Although the description and examples are restricted to the English language, each ambiguity exists in all languages that have the
Research method
The research method is to conduct several case studies, each involving examination of
- –
a real-world project, P, with a serious RE process, to develop a CBS, C,
- –
the project's RS, R, specifying C, and
- –
the project's implementation, I, of C according to R,
That the case study be of a real-world project means that it would be necessary to obtain R from a real-world company, , that is developing CBSs. In
Conduct of studies
We were able to find a company that was willing to provide RSs for already-completed CBS developments. To obtain the company's documents and cooperation in conducting the case studies, we had to sign a non-disclosure agreement, i.e., not to reveal the company's name, business, and RSs. Consequently, none of the examples in this paper of ambiguities are from the RSs. To allow the reader to understand the nature of the RSs, we can say that the company's business involves serving customers that
Results
For each of most ambiguity types, the results of each of the RSs are discussed separately because different requirements engineers were involved for each project. Also, for each ambiguity type, an analysis of patterns and differences between the three projects is provided. The projects are referred to as “P1”, “P2”, and “P3”, and their RSs are R1, R2, and R3, respectively.
Final verdict
The concerted attempt of the case studies to falsify de Bruijn's and Philippo's conclusions failed, and the results of the case studies end up supporting their conclusion, namely that ambiguities in an RS are not a serious problem for a well-run CBS development because in-project discussions about the CBS's requirements ends up resolving all ambiguities either consciously or unconsciously. In fact, in the case studies, none of the ambiguities that the CREs did not know about exposed any defects
Discussion
This section attempts to use the results to answer the RQs:
- 1.
What is the prevalence of PA?
- 2.
What is the impact of PA?
Conclusions
Current research is questioning the value of inspecting RSs for ambiguity because it has been shown by de Bruijn et al. [11] and Philippo et al. [12] in two separate case studies that during normal serious RE that leads to an RS, ambiguities in the RS are resolved through the continual discussion that takes place normally during the RE. The purpose of the research described in this paper was to try the hardest to falsify de Bruijn's and Philippo's results, each of which is based on a random
Acronyms
- CRE
chief requirements engineer
- CBS
computer-based system
- NL
natural language
- PA
persistent ambiguity
- RE
requirements engineering
- RS
requirements specification
- RQ
research question
- SDLC
software development lifecycle
- SA
subconscious ambiguation
- SD
subconscious disambiguation
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
Cristina Ribeiro's and Daniel Berry's work were supported in parts by a Canadian NSERC Discovery Grant NSERC-RGPIN227055.
References (58)
The Prevalence and Impact of Persistent Ambiguity in Software Requirements Specification Documents
(2016)- et al.
Mastering the Requirements Process
(2006) - et al.
Exploring Requirements: Quality Before Design
(1989) Practical Software Requirements: A Manual of Content and Style
(1998)- et al.
Attempto controlled english (ACE) language manual version 3.0
(1999) - et al.
Ambiguity measures in requirement engineering
- et al.
From contract drafting to software specification: Linguistic sources of ambiguity
(2003) - et al.
Ambiguity in requirements specification
Ambiguity in natural language requirements documents
User DRIVEN design—the luxury that has become a necessity