Experience report: Can software testing education lead to more reliable code?

Software Testing (ST) is one of the least known aspects of software development. Yet, software engineers often argue that it demands more than half of the costs of a software project. Thus, proper testing education is of paramount importance. In fact, the mere exposition to ST knowledge might have an impact on programming skills. In particular, it can encourage the production of more reliable code. Although this is intuitive, to the best of our knowledge, there are no empirical studies about such effects. Evidence on this matter is important to motivate - or demotivate - classical testing education. Concerned with this, we have conducted a study to investigate the possible impact of ST knowledge on the production of reliable code. Our controlled experiment involved 28 senior-level Computer Science students, 8 auxiliary functions with 92 test cases, and a total of 112 implementations. Results show that code delivered after the exposition to ST knowledge is, on average, 20% more reliable (a significant difference at the 0.01 level). Also, implementations delivered afterwards are not significantly larger in terms of lines of code. This indicates that ST knowledge can make developers produce more reliable software with no additional overhead in terms of program size.