Lessons from using Z to specify a software tool

The authors were recently involved in the development of a COBOL parser (G. Ostrolenk et al., 1994), specified formally in Z. The type of problem tackled was well suited to a formal language. The specification process was part of a life cycle characterized by the front loading of effort in the specification stage and the inclusion of a statistical testing stage. The specification was found to be error dense and difficult to comprehend. Z was used to specify inappropriate procedural rather than declarative detail. Modularity and style problems in the Z specification made it difficult to review. In this sense, the application of formal methods was not successful. Despite these problems the estimated fault density for the product was 1.3 faults per KLOC, before delivery, which compares favorably with IBM´s Cleanroom method. This was achieved, despite the low quality of the Z specification, through meticulous and effort intensive reviews. However, because the faults were in critical locations, the reliability of the product was assessed to be unacceptably low. This demonstrates the necessity of assessing reliability as well as “correctness” during system testing. Overall, the experiences reported in the paper suggest a range of important lessons for anyone contemplating the practical application of formal methods