Selective freezing of impaired video frames using low-rate shift-invariant hint

Irrecoverable data loss may be unavoidable for real-time video communication over common best-effort networks. Rather than always display impaired pictures or always “freeze” the last good picture, it is preferable to transmit additional hints to support selective freezing of heavily damaged pictures only. In particular, errors in impaired pictures tend to be localized, and often manifest themselves as small spatial shifts that are visually preferable to freezes. Therefore, it is essential that the hints can identify localized error and do not penalize small shifts. We show two ways such “shift-invariant” hints for detecting concealment error can be constructed to less than 1% of video rate. Experiments using 720p sequences achieve recall and precision of 90% and 75%, respectively, with respect to a shift-invariant PSNR measure. We also present an adaptive decision rule to obtain shorter and less frequent freezes.