Automatic generation of test oracles: from pilot studies to application

There is a trend towards the increased use of automation in V&V (verification and validation). Automation can yield savings in time and effort. For critical systems, where thorough V&V is required, these savings can be substantial. We describe a progression from pilot studies to development and use of V&V automation. We used pilot studies to ascertain opportunities for, and suitability of automating various analyses whose results would contribute to V&V. These studies culminated in the development of an automatic generator of automated test oracles. This was then applied and extended in the course of testing an AI planning system that is a key component of an autonomous spacecraft