Adaptive critic learning with fuzzy utility

Adaptive critic methods, which approximate dynamic programming, have been used successfully for solving optimal control problems. The adaptive critic learning algorithm optimizes a secondary utility function that is the sum of the present and all future primary utility. The primary utility function measures the instantaneous cost incurred for the last action taken and the resulting state. The motivation for using a fuzzy primary utility function comes from the set of control problems for which there is only a qualitative definition of performance - for example, success or failure. Previous work in applying adaptive critic methods to this type of problem showed that a crisp definition of success resulted in solutions that met the control objective, but in an undesirable manner. An appropriate fuzzy utility function, on the other hand, is able to generate the optimal solution. Another motivation for incorporating fuzzy techniques into the utility function is to overcome measurement noise. Measurement noise has a significant adverse effect on the reliability and speed of adaptive critic learning; by incorporating fuzzy sets into the utility function, the effect of the noise can be mitigated.