Title :
A Martingale Framework for Detecting Changes in Data Streams by Testing Exchangeability
Author :
Ho, Shen-Shyang ; Wechsler, Harry
Author_Institution :
Center for Automated Res., Univ. of Maryland, College Park, MD, USA
Abstract :
In a data streaming setting, data points are observed sequentially. The data generating model may change as the data are streaming. In this paper, we propose detecting this change in data streams by testing the exchangeability property of the observed data. Our martingale approach is an efficient, nonparametric, one-pass algorithm that is effective on the classification, cluster, and regression data generating models. Experimental results show the feasibility and effectiveness of the martingale methodology in detecting changes in the data generating model for time-varying data streams. Moreover, we also show that: (1) An adaptive support vector machine (SVM) utilizing the martingale methodology compares favorably against an adaptive SVM utilizing a sliding window, and (2) a multiple martingale video-shot change detector compares favorably against standard shot-change detection algorithms.
Keywords :
data analysis; management of change; pattern classification; regression analysis; stochastic processes; support vector machines; time-varying systems; adaptive support vector machine; change detection; martingale framework; martingale video shot change detector; one pass algorithm; regression data generating model; shot change detection algorithm; sliding window; testing exchangeability; time varying data stream; Change detection; classification; clustering; data stream; exchangeability; hypothesis testing; martingale; regression; support vector machine.;
Journal_Title :
Pattern Analysis and Machine Intelligence, IEEE Transactions on
DOI :
10.1109/TPAMI.2010.48