Spatial Reference Resolution for an Embodied Dialogue Agent

Abstract

In dialogue systems, reference resolution is an essential part of the semantic interpretation of an utterance. The resolution task is a variable-assignment problem – given the user’s utterance, we want to connect unknown discourse en-tities to our system’s internal knowledge base. To illustrate this problem, consider the following example sentences: (1) A: Mary likes John. B: I heard that he gave her a rose. In order to understand the meaning of these sentences, we must solve two resolution problems. The first sentence introduces two entities to our domain, Mary and John and can be semantically represented as likes(Mary, John). In the second sentence, we encounter the two pronouns he and her. The task is to then resolve the variables he to John and her to Mary, providing the appropriate semantics of the sentence. [gave(John, rose, Mary)] Dialogue systems like COMMUNICATOR, TRIPS and TRAINS all include some manner of reference resolution (Byron & Allen 1998). These traditional systems only con-sider elements of the discourse history and planning state as referents for an ambiguous expression. Extra-linguistic information, such as gesturing or spatial orientation, is ig-nored. Our work looks to extend this previous work to in-clude spatial context in reference resolution. This is a natural extension of previous work, and holds potential in embedded applications (cars, houses) and robotics. Spatial Reference Resolution Context of an ambiguous expression is not limited to just language. We often refer to an item by its spatial location. A host might tell his guest that “the spoons are in the right drawer. ” In resolving this reference, the guest must figure out how the description “right ” relates to the spatial context and aids in picking out the intended referent. Using spatial context for reference resolution requires both knowledge of how spatial relationships are organized and also how they are encoded linguistically. Linguistic investigation into spatial reference (Levinson 2003) has Copyright c © 2007, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. yielded three related strategies of encoding spatial refer-ence. To illustrate, consider the following sentences, each referring to a particular door in the world: