Title :
Policy optimisation of POMDP-based dialogue systems without state space compression
Author :
Gasic, M. ; Henderson, Mike ; Thomson, B. ; Tsiakoulis, Pirros ; Young, Stephanie
Author_Institution :
Eng. Dept., Cambridge Univ., Cambridge, UK
Abstract :
The partially observable Markov decision process (POMDP) has been proposed as a dialogue model that enables automatic improvement of the dialogue policy and robustness to speech understanding errors. It requires, however, a large number of dialogues to train the dialogue policy. Gaussian processes (GP) have recently been applied to POMDP dialogue management optimisation showing an ability to substantially increase the speed of learning. Here, we investigate this further using the Bayesian Update of Dialogue State dialogue manager. We show that it is possible to apply Gaussian processes directly to the belief state, removing the need for a parametric policy representation. In addition, the resulting policy learns significantly faster while maintaining operational performance.
Keywords :
Bayes methods; Gaussian processes; Markov processes; decision theory; interactive systems; speech processing; Bayesian update; GP; Gaussian processes; POMDP dialogue management optimisation; belief state; dialogue policy; dialogue state dialogue manager; parametric policy representation; partially observable Markov decision process; policy optimisation; speech understanding errors; Approximation methods; Bayesian methods; Error analysis; Gaussian processes; Kernel; Optimization; Training; Gaussian process; POMDP; statistical dialogue modelling;
Conference_Titel :
Spoken Language Technology Workshop (SLT), 2012 IEEE
Conference_Location :
Miami, FL
Print_ISBN :
978-1-4673-5125-6
Electronic_ISBN :
978-1-4673-5124-9
DOI :
10.1109/SLT.2012.6424165