An Online Clustering Algorithm That Ignores Outliers: Application to Hierarchical Feature Learning from Sensory Data

Surveillance sensors are a major source of unstructured Big Data. Discovering and recognizing spatiotemporal objects (e.g., events) in such data is of paramount importance to the security and safety of facilities and individuals. Hierarchical feature learning is at the crux to the problems of discovery and recognition. We present a multilayered convergent neural architecture for storing repeating spatially and temporally coincident patterns in data at multiple levels of abstraction. The bottom-up weights in each layer are learned to encode a hierarchy of over complete and sparse feature dictionaries from space- and time-varying sensory data by recursive layer-by-layer spherical clustering. This density-based clustering algorithm ignores outliers by the use of a unique adaptive threshold in each neuron´s transfer function. The model scales to full-sized high-dimensional input data and also to an arbitrary number of layers, thereby possessing the capability to capture features at any level of abstraction. It is fully-learnable with only two manually tunable parameters. The model was deployed to learn meaningful feature hierarchies from audio, images and videos which can then be used for recognition and reconstruction. Besides being online, operations in each layer of the model can be implemented in parallelized hardware, making it very efficient for real world Big Data applications.