Ongoing quality improvement, or: how we all learned to trust XP

Since VA Software adopted XP 2 1/2 years ago, the understanding of quality and the product quality itself have changed dramatically. Before we started with XP our code had alpha quality. Features were tolerably implemented, the UI was very bad and documentation was not even started. Today, the complete product (code, UI, documentation, SDKs) is constantly at pre-GA level. At any point in our development we can provide prereleases to our customers without any additional testing or release effort. Before we adopted XP it took us up to 1 1/2 man years (6 QA engineers for 3 months) to bring a product release from development end to release. Our first XP release required 6 man months (4 QA engineers for 6 weeks). Now we are down to 4.5 man weeks (1 1/2 QA engineers for 3 weeks). For experienced XP practitioners it might seem strange that the quality did not immediately jump to the high level that we have today after adopting XP. This was caused by a scenario that seems to be very typical for companies that have an established waterfall based development process: the engineering team was enthusiastic about the new way of development, the product management team was indifferent and the QA team was highly skeptical of the promise of XP to generate an (almost) bug-free product without a long and manual testing effort after development was done. Nobody in the engineering or product management team had any experience with XP, so it was impossible to convince the QA team otherwise. (In fact, reports of bug-free code from other XP teams were more intimidating than inspiring). So, we started our XP process without full QA involvement and a full system test phase after development was done. This paper recounts the evolution of our development process. It shows all the phases that our process went through, major changes that we made and what the overall impact was.