Improving Personalized Ranking in Recommender Systems with Multimodal Interactions

This paper proposes a conceptual framework which uses multimodal user feedback to generate a more accurate personalized ranking of items to the user. Our technique is a response to the actual scenario on the Web, where users can consume content following different interaction paradigms, such as rating, browsing, sharing, etc. We developed a post-processing step to ensemble rankings generated by unimodal-based state-of-art algorithms, using a set of heuristics which analyze the behavior of the user during consumption. We provide an experimental evaluation using the Movie Lens 10M dataset, and the results show that better recommendations can be provided when multimodal interactions are considered for profiling the preferences of the users.