A framework for automatic large-scale testing and characterization of signal processing algorithms

Performance analysis of signal processing algorithms should yield insight into expected performance as a function of all varying factors including algorithm parameters, signal characteristics, and transmission channel propagation effects. This paper presents a test framework for characterizing signal processing algorithm behavior. The framework provides functions to automate the evaluation process including creating large numbers of test files, running these files through the algorithm under test, collecting data, and analyzing the results. Automating this cycle allows rapid testing and evaluation of algorithm enhancements, as well as identifying the significant factors that affect performance. We demonstrate this technique by comparing and characterizing two different published symbol rate estimation algorithms.