Planning Multimedia Explanations Using Communicative Acts

Abstract

A number of researchers have investigated the use of planbased approaches to generate textual explanations (e.g., Appelt 1985; Hovy 1988; Moore 1989; Maybury 1990b). This paper extends this approach to generate multimedia explanations by defining three types of communicative acts: linguistic acts (illocutionary and locutionary speech acts), visual acts (e.g., deictic acts), and media-independent rhetorical acts (e.g., identify, describe). This paper formalizes several of these communicative acts as operators in the library of a hierarchical planner. A computational implementation is described which uses these plan operators to compose route plans in coordinated natural language and graphics in the context of a cartographic information system.