Designing Comprehensible Agents

Abstract

For many applications, it is important that agents be not only correct, but also compre-hensible to human users. Typically, people have tr ied to make agents ' behavior and reason-ing understandable by adding post-hoc special-purpose explanation systems, wi th often dis-appointing results. Here, I instead take the comprehensibility of agent behavior as a cen-t ra l agent design consideration from the start. I describe an agent architecture, the Expressiva-tor, that supports comprehensibility on top of a behavior-based framework, using four technical innovations: (1) structuring the agent's behav-ior according to the signs and signifiers it is in-tended to communicate; (2) allowing the agent