Effect of variability of a framework upon its testing effort: An empirical evaluation

Software frameworks, common and reusable to the development of a family of applications, are pre-implemented architectures having predefined points of variability. During framework reuse, implementation of one (or more than one) of the different possible alternatives of the variability take place. Variability is the number of possible variant implementations of hook methods, which are actually abstract or empty methods, in the framework. The reusability (number of opportunities of reuse) of a framework increases with the increase in abstractness or variability of the framework. High reusability is desired and therefore a framework developer may go on increasing the variability of a framework by providing more loose points, i.e. hot spots, to it. In the present paper, an experimental analysis has been carried out to understand the effect of variability possessed by a framework on the effort required to test the framework and it has been found that although the reusability of a framework increases with increase in the variability/abstractness of a framework but it does have adverse effects upon the testability of the framework. Hence, as a framework developer one has to restrain increasing the abstractness of a framework (while developing it) in order to put constraints over testing effort. The reusable framework demands stricter and rigorous testing in comparison to one-time use software artefacts. It is always advisable to guaranty the production of high quality reusable artefacts without incurring heavy costs for rigorous testing.