The Mutating Metric for Benchmarking Test

The monitoring of three ATPG tool parameters-fault coverage, test generation time, and test vector count-has improved test benchmarking over the years. Changing test dynamics, however, requires a different test metric. Over the past two decades, test automation has evolved from manually created patterns for designs with fewer than 100 gates to automated test pattern generation for designs with 10 million gates. At every stage, a benchmark has provided a comparison point for the technology. Benchmarks have played a significant role in improving test automation, and a metric appropriate to the state of the art at any given moment quantifies the technology on benchmarks. Metrics change as technology changes, and test has seen its own share of changes to the metrics. In this article, we examine the evolution of metrics and propose a metric for future evaluations of test technology