Title :
Data-Driven Affective Filtering for Images and Videos
Author :
Li, Teng ; Ni, Bingbing ; Xu, Mengdi ; Wang, Meng ; Gao, Qingwei ; Yan, Shuicheng
Author_Institution :
College of Electrical Engineering and Automation, Anhui University, Hefei, China
Abstract :
In this paper, a novel system is developed for synthesizing user-specified emotions onto arbitrary input images or videos. Other than defining the visual affective model based on empirical knowledge, a data-driven learning framework is proposed to extract the emotion-related knowledge from a set of emotion-annotated images. In a divide-and-conquer manner, the images are clustered into several emotion-specific scene subgroups for model learning. The visual affection is modeled with Gaussian mixture models based on color features of local image patches. For the purpose of affective filtering, the feature distribution of the target is aligned to the statistical model constructed from the emotion-specific scene subgroup, through a piecewise linear transformation. The transformation is derived through a learning algorithm, which is developed with the incorporation of a regularization term enforcing spatial smoothness, edge preservation, and temporal smoothness for the derived image or video transformation. Optimization of the objective function is sought via standard nonlinear method. Intensive experimental results and user studies demonstrate that the proposed affective filtering framework can yield effective and natural effects for images and videos.
Keywords :
Feature extraction; Hidden Markov models; Image color analysis; Semantics; Training; Videos; Visualization; Affective filtering; Gaussian mixture model (GMM); data-driven; linear piece-wise transformation;
Journal_Title :
Cybernetics, IEEE Transactions on
DOI :
10.1109/TCYB.2015.2392156