Semantic adaptation of consumer photo for mobile device access

This paper proposes a novel event-aware semantic image adaptation framework for accessing consumer photos via mobile devices. There are two major challenges to develop such an image adaptation framework: (1) how to bridge the semantic gap in the search for the desired photos from a consumer database; (2) how to perform appropriate adaptation for various small displays in order to maximize user´s perceptual experience according to the semantic significance of objects in photos. To meet the first challenge, an event semantics-guided extraction is developed based on statistical fusion of bottom-up low level features and top-down semantic features. To meet the second challenge, a key object-based semantic adaptation approach is designed to obtain the perceptually optimized Region of Interest (ROI). We conducted experiments based on the events defined in the Kodak consumer photo database. These experiments show that by utilizing the semantics of the photos with event-based a priori knowledge, the adaptation result outperforms the conventional attention based scheme. More importantly, the proposed framework can be easily extended to other events by adopting the event semantics based a priori knowledge into the adaptation system.