Autonomous novelty detection and object tracking in video streams using evolving clustering and Takagi-Sugeno type neuro-fuzzy system

Autonomous systems for surveillance, security, patrol, search and rescue are the focal point of extensive research and interest from defense and the security related industry, traffic control and other institutions. A range of sensors can be used to detect and track objects, but optical cameras or camcorders are often considered due to their convenience and passive nature. Tracking based on color intensity information is often preferred than the motion cues due to being more robust. The technique presented in this paper can also be used in conjunction with infra-red cameras, 3D lasers which result in a grey scale image. Novelty detection and tracking are two of the key elements of such systems. Most of the currently reported techniques are characterized by high computational, memory storage costs and are not autonomous because they usually require a human operator in the loop. This paper presents new approaches to both the problem of novelty detection and object tracking in video streams. These approaches are rooted in the recursive techniques that are computationally efficient and therefore potentially applicable in real-time. A novel approach for recursive density estimation (RDE) using a Cauchy type of kernel (as opposed to the usually used Gaussian one) is proposed for visual novelty detection and the use of the recently introduced evolving Takagi-Sugeno (eTS) neuro-fuzzy system for tracking the object detected by the RDE approach is proposed as opposed to the usually used Kalman filter (KF). In fact, eTS can be seen as a fuzzily weighted mixture of KF. The proposed technique is significantly faster than the well known kernel density estimation (KDE) approach for background subtraction for novelty detection and is more precise than the usually used KF. Additionally the overall approach removes the need of manually selecting the object to be tracked which makes possible a fully autonomous system for novelty detection and tracking to be developed.