DocumentCode :
11677
Title :
Gaussian Processes for POMDP-Based Dialogue Manager Optimization
Author :
Gasic, M. ; Young, Stephanie
Author_Institution :
Dept. of Eng., Univ. of Cambridge, Cambridge, UK
Volume :
22
Issue :
1
fYear :
2014
fDate :
Jan. 2014
Firstpage :
28
Lastpage :
40
Abstract :
A partially observable Markov decision process (POMDP) has been proposed as a dialog model that enables automatic optimization of the dialog policy and provides robustness to speech understanding errors. Various approximations allow such a model to be used for building real-world dialog systems. However, they require a large number of dialogs to train the dialog policy and hence they typically rely on the availability of a user simulator. They also require significant designer effort to hand-craft the policy representation. We investigate the use of Gaussian processes (GPs) in policy modeling to overcome these problems. We show that GP policy optimization can be implemented for a real world POMDP dialog manager, and in particular: 1) we examine different formulations of a GP policy to minimize variability in the learning process; 2) we find that the use of GP increases the learning rate by an order of magnitude thereby allowing learning by direct interaction with human users; and 3) we demonstrate that designer effort can be substantially reduced by basing the policy directly on the full belief space thereby avoiding ad hoc feature space modeling. Overall, the GP approach represents an important step forward towards fully automatic dialog policy optimization in real world systems.
Keywords :
Gaussian processes; Markov processes; human computer interaction; interactive systems; optimisation; speech processing; GP policy optimization; Gaussian processes; POMDP-based dialogue manager optimization; ad hoc feature space modeling; automatic dialog policy optimization; full belief space; learning process; partially observable Markov decision process; policy representation; real-world dialog systems; speech understanding errors; user simulator; Approximation methods; Bayes methods; Gaussian processes; Kernel; Optimization; Speech; Speech processing; Gaussian process; POMDP; statistical dialog systems;
fLanguage :
English
Journal_Title :
Audio, Speech, and Language Processing, IEEE/ACM Transactions on
Publisher :
ieee
ISSN :
2329-9290
Type :
jour
DOI :
10.1109/TASL.2013.2282190
Filename :
6601004
Link To Document :
بازگشت