A Predictive Coding Framework for Learning to Predict Changes in Streaming Data

Streaming sensorial data poses major computational challenges, such as, lack of storage, inapplicability of offline algorithms, and the necessity to capture nonstationary data distributions with concept drifts. Our goal is to build a learner framework that uses the current data and the knowledge from historical data to predict the next data in an efficient, unsupervised and online manner. Labeled streaming data is scarce, hence prediction of data instead of labels is a more realistic problem. We present a learner model, called SELP, for learning in variances as features from explanations of surprises due to prediction errors in streaming spatiotemporal data. This model runs a relentless cycle of Surprise → Explain → Learn → Predict involving the real external world and its internal model. The learner is continuously updated, independent of a trigger, proportional to its surprise. It implements a more efficient version of predictive coding, a form of biologically-plausible information coding paradigm, by predicting changes in the data instead of the data itself. Experimental results obtained from deploying our implementation on synthesized and real-world data are qualitatively comparable to that of traditional predictive coding on similar data sets. The results also offer insights into the learner design. This research lays out the foundations for an agent-based framework with an internal model grounded to the data stream.