Empirical Evaluation of a Reinforcement Learning Spoken Dialogue System

Abstract

We report on the design, construction and empirical evaluation of a large-scale spoken dialogue system that optimizes its performance via reinforcement learning on human user dialogue data. Introduction The formalisms of Markov decision processes (MDPs) and reinforcement learning (RL) have become a standard approach to many AI problems that involve an agent learning to improve performance by interaction with its environment (Sutton, 1991; Kaelbling et al., 1996). While the theory of these formalisms is quite advanced, applications have been limited almost exclusively to problems in control, operations research, or game-playing (e.g., Crites and Barto, 1995; Tesauro, 1995). In this paper, we describe an application of RL to a rather different type of problem, in which the MDP models a system's interaction with a population of human users, and RL is used to optimize the system's performance. Strategy Dialogue Database TTS ASR User Figure 1: A block diagram representation of a ...