Using automatically generated descriptions of human activity to index multi-media data

Multi-media systems capture, generate, store and render data from a variety of non-textural media-not just video. Yet compared with the world of text, only a few relatively puny mechanisms for indexing and retrieval are provided. A method for automatically indexing any timestamped data set is described. It is being studied as part of an activity to build and evaluate a system to aid human recall, called the AIR project, (activity-based information retrieval). It is being designed to help people retrieve things from `memory´ that they would otherwise find difficult or impossible to retrieve, because, for example, they would not normally keep records on the subject. This system has been called a human memory prosthesis. It exploits a suggested autobiographical model of human memory which observes that one way that humans seem to organize their memory is in episodes. The AIR project team is interested in providing mechanisms for recalling a variety of types of data ranging from electronic or paper documents, information design notes, seminars, snatches of explanations, and impromptu `how to´ tutorials