Model-Based Testing of Video Conferencing Systems: Challenges, Lessons Learnt, and Results

Model-Based Testing (MBT) is a well-established and intense field of research in academia. It has attracted attention of many industries as it can be seen from many industrial experiences of MBT reported in the literature and availability of commercial and open source tools in recent years. The thorough and methodical approach of MBT facilitates automated testing with the intention of improving the quality of software systems. Every industrial application of MBT faces varied challenges depending on the application domain, the current testing practices and tools, and the type of testing. Reporting such challenges, their solutions, and lessons learnt provides a body of knowledge, which can direct practitioners of MBT for their future applications of MBT. With such aim in our mind, we present results from an MBT project that is being carried out for testing embedded video conferencing systems developed by Cisco Systems, Inc. Norway for the last several years. We present challenges faced while conducting MBT, our solutions, some of the key results, and lessons learnt from our experience. Our experience showed that search algorithms provide an efficient solution for test case selection and test data generation. In addition, aspect-oriented modeling provides a scalable modeling solution for non-functional testing. Finally, we learned that model transformation offers an elegant solution for developing a model-based test case generation tool. All of our results are based on a large number of rigorous empirical evaluations.