Multi-criteria simulation of program outcomes

Programs that develop and deploy complex systems typically have multiple criteria by which they are judged to be successful. Categories of such criteria include schedule, cost, technical system performance, quality and customer expectations. Criteria are operationalized via particular metrics, and often there are complex relationships between metrics, e.g., correlations or trade-offs. In an acquisition program, it is critical that systems engineers understand the implications of their actions and decisions with respect to these metrics, since the metrics are used to report the performance and eventual outcome of the program. However, such understanding usually takes many years of on-the-job experience. This paper describes an approach to simulation modeling of program behavior and performance whereby program outputs expressed in these metrics can be studied by systems engineers. An example program simulation model is presented that currently is used in an educational technology system for training systems engineers. The decisions and actions that can be taken by a systems engineer are described, and the impacts of various actions and decisions on program metrics and metric relationships are illustrated. The model is validated via subject matter experts with extensive experience in the program domain.