Quality Assessment of Mission Critical Middleware System Using MEMS

Architecture evaluation methods provide general guidelines to assess quality attributes of systems, which are not necessarily straightforward to practice with. With COTS middleware based systems, this assessment process is further complicated by the complexity of middleware technology and a number of design and deployment options. Efficient assessment is key to produce accurate evaluation results for stakeholders to ensure good decisions are made on system acquisition. In this paper, a systematic evaluation method called MEMS is developed to provide some structure to this assessment process. MEMS produces the evaluation plan with thorough design of experiments, definition of metrics and development of techniques for measurement. This paper presents MEMS and its application to a mission critical middleware system.