Inverse Reinforcement Learning for Interactive Systems

Abstract

Human machine interaction is a field where machine learning is present at almost any level, from human activity recognition to natural language generation. The interaction manager is probably one of the latest components of an interactive system that benefited from machine learning techniques. In the late 90's, sequential decision making algorithms like reinforcement learning have been introduced in the field with the aim of making the interaction more natural in a measurable way. Yet, these algorithms require providing the learning agent with a reward after each interaction. This reward is generally handcrafted by the system designer who introduces again some expertise in the system. In this paper, we will discuss a method for learning a reward function by observing expert humans, namely inverse reinforcement learning (IRL). IRL will then be applied to several steps of the spoken dialogue management design such as user simulation and clustering but also to co-adaptation of human user and machine.