Sequence recognition with spatio-temporal long-term memory organization

In this work, we propose a connectionist memory structure for spatio-temporal sequence learning and recognition inspired by the Long-Term Memory structure of human cortex. Besides symbolic data, our framework is able to continuously process real-valued multi-dimensional data stream. This capability is made possible by addressing three critical problems in spatio-temporal learning, namely error tolerance, significance of sequence´s elements and memory forgetting mechanism. We demonstrate the potential of the framework with a synthetic example and a real world example, namely the task of hand-sign language interpretation with the Australian Sign Language dataset.