Towards a Multimodal Framework for Human Behavior Recognition

This paper proposes a framework of an interface that is capable of detecting and classifying human intentions and personality types. The framework relies on a conventional set of multimodal components, namely an eye-tracking system, a mouse and a keyboard, where a large number of real time data analysis will take place, in order to detect behavioral patterns. The system gradually learns from the players and builds a collection of patterns of actions. The personalities of the players are detected through ontological comparisons of known personality types with the newly discovered patterns of actions. We further layout the ground of our initial assumption - appearing in other publications, that the sub-conscious controls the eyes movements during the game, on those elements or words that are related to the fears and desire, i.e., the personality of the player