Title :
An Audio and Image-Based On-Demand Content Annotation Framework for Augmenting the Video Viewing Experience on Mobile Devices
Author :
Gatteschi, Valentina ; Lamberti, Fabrizio ; Sanna, Andrea ; Demartini, Claudio
Author_Institution :
Dipt. di Autom. e Inf., Politec. di Torino, Turin, Italy
fDate :
June 27 2015-July 2 2015
Abstract :
The availability of annotated multimedia contents is a crucial requirement for a number of applications. In the context of education it could support the automatic summarization of recorded lessons or the retrieval of learning material. In the field of entertainment, it could serve to recommend audio and video resources based on user´s attitudes. In this work, a framework supporting video viewing experience augmentation on mobile devices by means of image- and text-based annotations extracted on-demand from Wikipedia is presented. Speech recognition is exploited to periodically get text snaps from the audio track of the video currently displayed on the mobile device, while query-by-images is used to generate a text summary of extracted video frames. Keywords obtained are treated by semantic techniques to find named entities associated with the multimedia contents, which are then superimposed to the video and displayed to the user in a synchronized way. Promising results obtained with a prototype implementation showed the feasibility of the proposed solution, which could be possibly combined with other systems, e.g., Providing information about user´s location, preferences, etc. To build up more sophisticated context-aware applications.
Keywords :
computer aided instruction; content management; entertainment; information retrieval; mobile computing; multimedia computing; speech recognition; video signal processing; Wikipedia; audio-based on-demand content annotation framework; automatic summarization; context-aware applications; education; entertainment; image-based on-demand content annotation framework; learning material retrieval; mobile devices; multimedia contents; speech recognition; video viewing; Conferences; Mobile communication; Wikipedia; multimedia content annotation; query by images; semantics; speech recognition;
Conference_Titel :
Mobile Services (MS), 2015 IEEE International Conference on
Conference_Location :
New York, NY
Print_ISBN :
978-1-4673-7283-1
DOI :
10.1109/MobServ.2015.71