Case study of PESQ performance in live wireless mobile VoIP environment

PESQ is the international standard for objective speech quality measurement and has been widely used in industry for voice quality assessment for VoIP products and systems. However, current research efforts involving PESQ are mainly focused on simulated IP networks. The need remains to establish how accurate PESQ is when used in real wireless systems, especially in new wireless VoIP environment. The main objective of this paper is to investigate the accuracy of PESQ measurement in live wireless mobile environment. We set up a PESQ performance evaluation platform and tested end-to-end voice quality for over 1800 mobile-to-mobile and mobile-to-PSTN calls over a period of 3 months. We found that there still exist some accuracy problems when using PESQ in mobile VoIP environment. These problems are mainly caused by improper time alignment in the PESQ algorithm when there are silence gap and speech sample removal or insertion due to packet loss concealment and jitter buffer adjustment in mobile devices. In the paper, we illustrate these problems in a detailed case study and provide suggestions for improvement of test performance.We also discussed the importance of calibration issues (e.g. volume optimization) for a safe and proper PESQ testing in mobile environment. The outcome of the study will be helpful in understanding problems associated with voice quality measurements in live mobile VoIP environment and how to further improve PESQ accuracy in mobile VoIP systems.