The Maintenance and Evolution of Field-Representative Performance Tests

The rise of large-scale software systems poses new challenges for the software performance engineering field. Failures in these systems are typically associated with performance issues, rather than with feature bugs. Therefore, performance testing has become essential to ensuring the problem-free operation of these systems. However, the performance testing process is faced with a major challenge: evolving field workloads and deployment configurations, often lead to "outdated" tests that are not reflective of the field. Hence performance analysts must maintain their field-representative tests by evolving these tests. Therefore, we propose new approaches for maintaining and evolving field-representative performance tests. We believe that field-representative tests can be autonomically maintained and evolved, i.e., the configurable parameters of a performance test can be optimized, such that the resulting test exercises the system in a similar manner as the field. Such approaches should help performance analysts to ensure the "realism" of their performance testing efforts, resulting in higher quality systems.