Focused Quality Assurance of Mobile Applications: Evaluation of a Failure Pattern Classification

Mobile applications require quality assurance approaches tailored to their peculiarities, such as context awareness and platform diversity. Furthermore, time limitations and budget restrictions in mobile application development are typical characteristics that need to be addressed. In a prior work, we proposed the usage of a mobile-specific failure pattern classification to focus quality assurance of mobile applications in order to achieve higher quality assurance effectiveness. In this contribution, we evaluate whether the proposed classification is reliable with regard to its effectiveness for finding quality issues, its completeness in terms of fault and failure classes, and its adequacy when used in a methodological approach. To this aim, we performed three experimental studies: a survey consulting experts of this field, a case study performed in industry, and project comparisons performed by analyzing issue tickets of several open-source mobile application projects. The overall results showed that the classification is reliable due to its effectiveness in finding failures of mobile applications during testing. However, each evaluation phase identified several possible extensions, which were added right after each evaluation study. Besides, the classification was confirmed as being well structured for use as part of a methodological approach.