When Is a Requirement Accurate?

Ivan Jureta

What conditions should a requirement satisfy, to be considered accurate? It turns out that this is a very complicated question. Two easy, yet unsatisfying ways out are (i) to think the question is irrelevant, and (ii) to claim that the validation of a requirement answers that question (i.e., if a stakeholder A gave the requirement R, asking A to confirm that they want R satisfied provides evidence that R is accurate). I argue that neither of these easy ways out are satisfactory, and that a different approach to answering the question yields interesting observations and additional questions.

What Is Accuracy?

In everyday usage, “accurate” is defined as “correct and without any mistakes” [1], whereby “correct” is “in agreement with the true facts or with what is generally accepted” [2] and mistake stands for “an action, decision, or judgment that produces an unwanted or unintentional result” [3].

If we look for a more rigorous definition, then the one from the BIPM (Bureau International des Poids et Mesures) is relevant [4]. It is for “measurement accuracy”; I also provide those of terms mentioned in the definition of “measurement accuracy”, namely “measured quantity value” and “true quantity value” and “measurand”.

2.13 measurement accuracy: closeness of agreement between a measured quantity value and a true quantity value of a measurand;

2.10 measured quantity value: quantity value representing a measurement result;

2.11 true quantity value: ​​quantity value consistent with the definition of a quantity;

2.3 measurand: quantity intended to be measured.

What is Accuracy of a Requirement?

In mainstream requirements engineering research, a requirement is an optative statement, that is a statement about what is desired [5,6]. This has two important implications for the present discussion:

  1. A requirement is a statement about something that may or may not be true when that statement is produced, that is, we can have these cases:
    1. a requirement refers to a present situation (or provides a partial description of a situation, as when we say that a requirement is a special kind of proposition [7]) that is true, and :
      1. is observable, so that there is a feasible way to observe that situation and ascertain its correspondence with what the requirement states, or
      2. is not observable, if there is no feasible way to establish if the conditions stated by the requirement indeed already hold;
    2. a requirement refers to a situation which does not presently hold, and it is desirable that it holds in the future, when that requirement is expected to be satisfied by putting in place some change / a solution to requirements, whatever exactly that solution is (software, hardware, some other artifact, an agreement, a process, responsibilities, and so on).
  2. A requirement is someone’s statement of what they desire (or what is desired by someone else, that those providing the statement have the authority to represent). It does not come from machines, or otherwise, but from people and reflects their desires, needs, expectations, intentions (as argued for a long time in newer mainstream requirements engineering – see [8,9,10,7] for example).

The first implication mentioned above boils down to the following distinction: observable satisfied requirement, unobservable satisfied requirement, and unsatisfied requirement. In all three cases, the second implication applies, that the requirement has an owner, the person who gave it. 

Since an unobservable satisfied requirement is, well, unobservable, i.e., we do not have a way to verify that it is satisfied, then it can be equated to the unsatisfied requirement. We can further simplify language, and drop “satisfied” from observable satisfied requirement, since it is observable because it is satisfied. We end up with two cases: 

  • observable requirement, and 
  • unsatisfied requirement. 

If we then take the more rigorous definition of accuracy, we need to ask what the true quantity value is. This leads to asking what “quantity” means when talking about a requirement.

Going back to BIPM, we have the following for “quantity”:

1.1 quantity: property of a phenomenon, body, or substance, where the property has a magnitude that can be expressed as a number and a reference

In the case of an observable requirement, the requirement refers to conditions that can be observed to hold. In other words, we know and can apply a process that yields a confirmation that the conditions referred to by the requirement are holding. 

In the case of observable requirement, accuracy refers to the difference between the description of conditions that the requirement provides, and the result of verifying that these conditions hold. 

In requirements engineering, a distinction between hard and soft (or non-functional) requirements is often made [11], where the satisfaction of a hard requirement is a binary variable, and the satisfaction of a soft requirement is a variable that can take more than two values. Therefore:

  • If the observable requirement refers to conditions that are either true or false, it is a hard requirement, and accuracy of that observable hard requirement is established by verifying if the conditions hold.
  • If the observable requirement is also a soft requirement, then matters may be more complicated, in that to claim accuracy, we need to define a measure whose values map to levels of satisfaction of the requirement, and a process and instrument to measure these. The most complicated cases involve soft requirements whose level of satisfaction is subjective, such as customer satisfaction with the product or service that supposedly observably satisfies a given soft requirement. 

In the case of an unsatisfied requirement, the requirement refers to conditions someone wants to see satisfied, but which are not presently satisfied. Here, there are in fact two further cases to distinguish:

  • unsatisfied requirement that is a negation of observable present conditions; this kind of unsatisfied requirement is called negative unsatisfied requirement below; and
  • unsatisfied requirement that is not a negation of current conditions, and may or may not relate in some other, more complicated way to current conditions; this is called the positive unsatisfied requirement below.

As noted above, in definitions from BIPM, accuracy is the distance between a true quantity and a measured quantity. What is the “true quantity” in the case of an unsatisfied requirement? What are, in other words, the conditions that accuracy is judged against?

For a negative unsatisfied requirement, “true quantity” or better here, “true desired conditions” is the negation of presently observable conditions. Negation is not necessarily such that it determines exactly what we should do, which causes more problems: if I say that I don’t want a red car, which color do I want? The default is that I simply don’t want a red car, and therefore, I want any other color than red, but perhaps if you let me think more, I might say black, which narrows down what I want relative to not wanting red. 

So the worst case is the case of positive unsatisfied requirements: they refer to conditions that do not hold now, and their negations may or may not hold now. 

What does it mean, then, that a positive unsatisfied requirement is accurate?

A positive unsatisfied requirement is accurate if and only if it reflects without error the conditions intended to be satisfied by the person who provided the requirement. 

The above is a problematic statement. We do not know the future. So when I provide a requirement that is a positive unsatisfied requirement, I intend to see some conditions satisfied in the future. But how well do I know these conditions, or better, how well can I know these conditions if I cannot observe them, or their negation today? Worse still, how well can I know them in a future that I predict, wheres the actual future that plays out may be different? Will my preferences still hold, and so I will still desire the same things, and thus the same conditions, as when I provided the requirement?

So it turns out that accuracy is relative to my intentions when providing requirements, a problem in itself, since my intentions are not observable [12]. Accuracy seems a lost cause for negative unsatisfied requirements.

The way to repair it, is to define, in addition to the requirement, a test of that requirement that unequivocally determines if the conditions stated in the requirement are satisfied. Somewhat unexpectedly, then, when we provide the requirement, accuracy is the distance between the requirement and the expected outcome of the test.

References

  1. https://dictionary.cambridge.org/dictionary/english/accurate
  2. https://dictionary.cambridge.org/dictionary/english/correct
  3. https://dictionary.cambridge.org/dictionary/english/mistake
  4. JCGM 200:2012 International vocabulary of metrology – BIPM
  5. Zave, Pamela, and Michael Jackson. “Four dark corners of requirements engineering.” ACM transactions on Software Engineering and Methodology (TOSEM) 6.1 (1997): 1-30.
  6. Gunter, Carl A., et al. “A reference model for requirements and specifications.” IEEE Software 17.3 (2000): 37-43.
  7. Jureta, Ivan, John Mylopoulos, and Stephane Faulkner. “Revisiting the core ontology and problem in requirements engineering.” 2008 16th IEEE International Requirements Engineering Conference. IEEE, 2008.
  8. Yu, Eric SK. “Towards modelling and reasoning support for early-phase requirements engineering.” Proceedings of ISRE’97: 3rd IEEE International Symposium on Requirements Engineering. IEEE, 1997.
  9. Van Lamsweerde, Axel. “Goal-oriented requirements engineering: A guided tour.” Proceedings fifth ieee international symposium on requirements engineering. IEEE, 2001.
  10. Dardenne, Anne, Axel Van Lamsweerde, and Stephen Fickas. “Goal-directed requirements acquisition.” Science of computer programming 20.1-2 (1993): 3-50.
  11. Chung, Lawrence, et al. Non-functional requirements in software engineering. Vol. 5. Springer Science & Business Media, 2012.
  12. Jureta, Ivan J. “What Happens to Intentional Concepts in Requirements Engineering If Intentional States Cannot Be Known?.” International Conference on Conceptual Modeling. Springer, Cham, 2017.