Robustness Testing of Mobile Telecommunication Systems: A Case Study on Industrial Practice and Challenges

Robustness relates to the capability of a system to handle internal and external negative situations and disturbances. Robustness testing is the act of subjecting the system under test to such disturbances in a controlled manner. The objective of this study is to understand how robustness is considered in the development and testing of large-scale telecom systems, to identify the main challenges related to robustness and robustness testing, and to identify potential improvements to the current situation. We performed an exploratory case study of a telecom industry. Data was collected through interviews, study of documentation, and participant observation. Our result is a number of challenges related to robustness testing. The key challenge identified relates to understanding how robustness test can be broken down from a system-level to a low-level perspective. Our conclusion is that the area of robustness test is challenging for large complex systems, where the understanding of how to provoke complex failures and derive a root cause, as well as defining the correct level of robust design in software is not sufficiently explored.