Deliberation scheduling using GSMDPs in stochastic asynchronous domains Original Research Article

We propose a new decision-theoretic approach for solving execution-time deliberation scheduling problems using recent advances in Generalized Semi-Markov Decision Processes (GSMDPs). In particular, we use GSMDPs to more accurately model domains in which planning and execution occur concurrently, plan improvement actions have uncertain effects and duration, and events (such as threats) occur asynchronously and stochastically. In this way, agents develop a continuous-time deliberation policy offline which can then be consulted to dynamically select deliberation-level and domain-level actions at plan execution-time. We demonstrate a significant improvement in expressibility over previous discrete-time approximate models in which mission phase duration was fixed, failure events were synchronized with phase transitions, and planning time was discretized into constant-sized planning quanta.