Search Based Testing of Embedded Systems Implemented in IEC 61131-3: An Industrial Case Study

This paper presents a case study of search-based test generation for embedded system software units developed using the Function Block Diagrams (FBDs), a graphical language in the IEC 61131-3 standard aimed at programmable logic controllers (PLCs). We consider 279 different components from the train control software developed by Bombardier Transportation, a major rail vehicle manufacturer. The software is compiled into C code with a particular structure. We use a modified hill climbing algorithm for generating test data to maximize MC/DC coverage for assignments with logical expressions in the C code, while retaining the semantics of the original FBD implementation. An experimental evaluation for comparing the effectiveness (coverage rate) and the efficiency (required number of executions) of hill climbing algorithm with random testing is presented. The results show that random testing performs well for most units under test, while around 30% of the artifacts significantly benefit from the hill climbing algorithm. Structural properties of the units that affect the performance of hill climbing and random testing are also discussed.