Using Spatial Language in Multi-Modal Knowledge Capture

Abstract

The ability to understand and communicate spatial relationships is central to many human-level reasoning tasks. People often describe spatial relationships using prepositions (i.e., in, on, under). Being able to use and interpret spatial prepositions could help create interactive systems for many tasks, including knowledge capture. Here I describe my thesis work modeling the learning and use of spatial prepositions and applying this model to the task of knowledge capture. Problem Being Addressed Spatial relationships play an important role in many reasoning tasks such as navigation and solving physics/engineering problems. Because space is such an important component of so many tasks, humans have developed specialized language for describing spatial relationships (i.e. prepositions such as in and on). Ideally, intelligent systems would be able to understand and use spatial language in their interactions with human users, particularly when doing visual-spatial tasks. Unfortunately, many systems modeling the use of spatial prepositions have had serious limitations. Many systems (e.g. Regier, 1995; Gap, 1995) operate only on geometric shapes, not real-world objects.