Learning probability distributions over partially-ordered human everyday activities

We propose a method to learn the partially-ordered structure inherent in human everyday activities from observations by exploiting variability in the data. Using statistical relational learning, the system extracts a full-joint probability distribution over the actions that form a task, their (partial) ordering, and their properties. Relevant action properties and relations among actions are learned as those that are consistent among the observations. The models can be used for classifying action sequences, for determining which actions are relevant for a task, which objects are usually manipulated, and which action properties are typical for a person. We evaluate the approach on synthetic data sampled from partial-order trees as well as two real-world data sets of humans activities: the TUM kitchen data set and the CMU MMAC data set. The results show that our approach outperforms sequence-based models like Conditional Random Fields for classifying observations of activities that allow a large amount of variation.