Scaling up Explanation Generation: Large-Scale Knowledge Bases and Empirical Studies

Abstract

To explain complex phenomena, an explanation system must be able to select information from a formal representation of domain knowledge, organize the selected information into multi-sentential discourse plans, and realize the dis-course plans in text. Although recent years have witnessed significant progress in the development of sophisticated computational mechanisms for explanation, empirical results have been limited. This paper reports on a seven year effort to em-pirically study explanation generation from se-mantically rich, large-scale knowledge bases. We first describe Knight, a robust explana-tion system that constructs multi-sentential and multi-paragraph explanations from the Biology Knowledge Base, a large-scale knowledge base in the domain of botanical anatomy, physiol-ogy, and development. We then introduce the Two Panel evaluation methodology and describe how Knight’s performance was assessed with this methodology in the most extensive empirical evaluation conducted on an explanation system. In this evaluation, Knight scored within “half a grade ” of domain experts, and its performance exceeded that of one of the domain experts. 1990; Hovy 1993; Maybury 1993) has developed tech-niques for determining the content and organization of many genres, and explanation generation (Cawsey 1992; Moore 1995) in particular has been the sub-ject of intense investigation. In addition to exploring a panorama of application domains, the explanation community has begun to assemble these myriad de-signs into a coherent framework. As a result, we have begun to see a crystallization of the major components (Suthers 1993).