A Systematic Mapping Study on Software Engineering Testbeds

Even though empirical research has grown in interest, techniques, methodologies and best practices are still in debate. In this context, test beds are effective when one needs to evaluate and compare technologies. The concept is well disseminated in other areas such as Computer Networks, but remains poorly explored in Software Engineering (SE). This paper presents a systematic mapping study on the SE test beds literature. From the initial set of 4239 studies, 13 primary studies were selected and categorized. Based on that, we found that Software Architecture is the most investigated topic, controlled experiment is the most used method to evaluate such test beds, 20 benefits of using test beds in SE have been identified and that test beds comprise very heterogeneous structural elements.