Is A Magnetic Sensor Capable of Evaluating A Vision-Based Face Tracking System?

This paper addresses an important issue, how to evaluate a vision-based face tracking system? Although nowadays it is getting popular to employ a magnetic sensor to evaluate the performance of such systems. The related issues such as condition and limitation of usage are often omitted. In this paper we studied this accepted evaluation methodology together with another evaluation method, Peak Signal to Noise (PSNR) commonly used in image coding community. The condition of proper usage of magnetic sensor as evaluating system is discussed. Our theoretical analysis and experiments with real video sequences show that we have to be very careful to select the so-called "ground truth". We believe that to help further development of face tracking techniques, a valid performance evaluation is necessary, both the evaluating system and the tracking system have to be jointly considered to decide if the evaluating method is valid. The experimental results give us further hints about the tracking performance when using different tracking scheme.