Online learning classifiers in dynamic environments with incomplete feedback

In this paper we investigate the performance of XCSR (a real-valued genetics-based machine learning method) in an online environment in which the feedbacks are received with a delay and not for all the instances. The importance of such environments lies in the fact that many real world environments have these characteristics. For instance, in spam detection some of the undetected spam messages which are delivered to the user may be flagged as spam by user after a while. Hence, the feedback is both delayed and partial in this context. Similar situation can easily be imagined in other fraud detection contexts such as network intrusion and credit card fraud. We also present an architecture for an adaptable online XCSR and present two heuristics to deal with biased partial feedback environments. The heuristics use the information about the environment and their observations and create artificial feedbacks for the classifications that do not receive any feedback. We show that these heuristics always help XCSR learn better and perform more accurately in such situations.