Evaluating requirements modeling methods based on user perceptions: a family of experiments

Abstract

Numerous methods and techniques have been proposed for requirements modeling, although very few have had widespread use in practice. One drawback of requirements modeling methods is that they lack proper empirical evaluations. This means that there is a need for evaluation methods that consider both the theoretical and practical aspects of this type of methods and techniques. In this paper, we present a method for evaluating the quality of requirements modeling methods based on user perceptions. The evaluation method consists of a theoretical model that explains the relevant dimensions of quality for requirements modeling methods, along with a practical instrument with which to measure these quality dimensions. Basically, it allows us to predict the acceptance of a particular requirements modeling method in practice, based on the effort of applying the method, the quality of the requirements artifacts produced, and the user perceptions with regard to the quality of the method. The paper also presents an empirical test of the proposed method for evaluating a Rational Unified Process (RUP) extension for requirements modeling. That test was carried out through a family of experiments conducted with students and practitioners and provides evidence of the usefulness of the evaluation method proposed. © 2011 Elsevier Inc. All rights reserved.