A precision-weighted rank-ordering procedure for the combination of voice coder evaluation results

A precision-weighted rank-ordering procedure was developed for the combination of voice coder subjective evaluation results to provide a final measure of performance for the reference and candidate coders in the NATO 1200/2400 bit/s coder selection project. The combination procedure is based on calculated error propagation, and precision-weighted rank-ordering. The procedure was necessary because the speech performance measurements were of several speech characteristics and used a variety of test methods conducted at multiple laboratories in different languages. The procedure allows for the definition of weights that represent the design criteria by which the coder matches its intended applications. The procedure also imposes implicit weights that are a function of the precision of the individual test methods.