Reinforcement learning in case-based systems

Case-based reasoning (CBR) systems compare a new problem to a library of cases and adapt a similar library case to the problem, producing a preliminary solution. Since CBR systems require only a record of library cases with successful solutions, they are often used in areas lacking a strong theoretical domain model, such as medicine, economics and law. The problem is that many CBR systems use expert knowledge to determine how to build indexes for the case library so that cases that match the current situation can be identified. Human experts - whose time is valuable and scarce - often find it difficult to explain precisely their reasoning. Thus, a knowledge elicitation bottleneck occurs for many knowledge-based applications. Reinforcement-trained case-based reasoning (RETCBR) uses feedback from the user or some external process to learn how to match cases. RETCBR expands the domains in which CBR techniques can be applied, because it requires knowledge only for case recognition, not for determining indexing strategies. We are currently applying these techniques to human-computer interaction problems, such as user modeling and collaborative filtering.