Robust test design for reliability estimation with modeling error when combining full system and subsystem tests

This paper develops a method for finding a robust test plan, which consists of a mixture of full system and subsystem tests, to estimate the reliability of a system given that the model for the system reliability is flawed. A robust test plan is developed by trading off the number of full system and subsystem tests to minimize the mean-squared error (MSE) of the maximum likelihood estimate (MLE) of system reliability. The MSE is decomposed into the variance of the MLE and a bias from incorrectly specifying the model that relates the subsystem reliabilities to the full system reliability (series, parallel, other). The variance of the MLE is based on the inverse Fisher Information. The bias is due to the modeling error. A demonstration of the test plan determination is given for a hypothetical system by trading off between the MSE (estimation accuracy), the degree of modeling error, and the cost of doing system and subsystem tests. A Pareto frontier can be identified, as illustrated in the paper.