Parallel program performance metrics: a comparison and validation

The authors present a novel technique, called true zeroing, that permits direct, quantitative, and fair comparison of parallel program performance metrics. This technique was applied to three programs that include both numeric and symbolic applications. Three existing metrics, Gprof, Critical Path, and Quartz/NPT, and several new variations were compared. The result of this comparison was that while Critical Path provided the best overall guidance, it was not universally better than the other metrics. Because there is no single universal metric, future parallel performance systems need to support multiple metrics. The authors present a set of recommendations to tool builders based on the experience gained during this case study