Measuring the effectiveness of computer based systems: an open system measurement example

Openness is one quality which modern computer based systems (CBSs) strive to possess. This work on the evaluation of CBSs was developed in the context of research work to measure open systems. Openness in large computer based systems is a quality sought by many, claimed by many, but proven by few. Openness in many systems with which we have daily contact is a quality which makes those systems operate in a fair and effective manner. The paper explores ways in which the relative openness of large computer based systems may be measured reliably, repeatedly and fairly. The aim of the project presented was to determine the relative openness between various architectures, and the (long-term) consequences of utilising various architectures. Along the way, it has provided significant evidence of the usefulness of the open systems approach for large computer based systems. The paper certainly provides a basis upon which to judge whether or not a system which claims to be open lives up to the claims of being a cost effective, evolvable complex computer based system. The outcome is a measurement system which should be able to deal with all practical propositions, is based on solid evidence and provides solid analytical measures which are expressed in plain language (outlining consequences). A most important outcome of the paper is beginning to establish whether or not openness is of any use.