An annotation-free method for evaluating privacy protection techniques in videos

While several privacy protection techniques are presented in the literature, they are not complemented with an established objective evaluation method for their assessment and comparison. This paper proposes an annotation-free evaluation method that assesses the two key aspects of privacy protection that are privacy and utility. Unlike some existing methods, the proposed method does not rely on the use of subjective judgements and does not assume a specific target type in the image data. The privacy aspect is quantified as an appearance similarity and the utility aspect is measured as a structural similarity between the original raw image data and the privacy-protected image data. We performed an extensive experimentation using six challenging datasets (including two new ones) to demonstrate the effectiveness of the evaluation method by providing a performance comparison of four state-of-the-art privacy protection techniques.