DocumentCode :
2061031
Title :
A unified framework for linear function approximation of value functions in stochastic control
Author :
Sanchez-Fernandez, Matilde ; Valcarcel, Sergio ; Zazo, S.
Author_Institution :
Signal Theor. & Communictions Dept., Univ. Carlos III de Madrid, Leganes, Spain
fYear :
2013
fDate :
9-13 Sept. 2013
Firstpage :
1
Lastpage :
5
Abstract :
This paper contributes with a unified formulation that merges previous analysis on the prediction of the performance (value function) of certain sequence of actions (policy) when an agent operates a Markov decision process with large state-space. When the states are represented by features and the value function is linearly approximated, our analysis reveals a new relationship between two common cost functions used to obtain the optimal approximation. In addition, this analysis allows us to propose an efficient adaptive algorithm that provides an unbiased linear estimate. The performance of the proposed algorithm is illustrated by simulation, showing competitive results when compared with the state-of-the-art solutions.
Keywords :
Markov processes; function approximation; signal processing; stochastic systems; Markov decision process; adaptive algorithm; linear estimate; linear function approximation; stochastic control; unified framework; value functions; Approximation algorithms; Cost function; Equations; Function approximation; Linear approximation; Mathematical model; Approximate dynamic programming; Linear value function approximation; Mean squared Bellman Error; Mean squared projected Bellman Error; Reinforcement Learning;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Signal Processing Conference (EUSIPCO), 2013 Proceedings of the 21st European
Conference_Location :
Marrakech
Type :
conf
Filename :
6811729
Link To Document :
بازگشت