Active Learning for Pipeline Models

Abstract

Decomposing complex classification tasks into a series of sequential stages, where the local classifier at each stage is explicitly dependent on predictions from previous stages, is a common practice. In the machine learning and natural language processing communities, this widely used paradigm is generally referred to as a pipeline model. This approach has been successfully applied to many tasks, including parsing, semantic role labeling, and textual entailment [3]. The primary motivation for modeling complex tasks as a pipelined process is the difficulty of solving such applications with a single classifier; that learning a classifier for a problem such as relation extraction directly in terms of input text may be impossible with the given resources. A second feature of domains requiring such decompositions is the corresponding high cost associated with obtaining sufficient labeled data. The active learning protocol offers one promising solution to this dilemma by allowing the learning algorithm to incrementally select unlabeled examples for labeling by the domain expert with the goal of maximizing performance while minimizing supervision [1]. While receiving significant recent attention, most active learning research focuses on new algorithms as they relate to a single classification task. This work instead assumes that an active learning algorithm exists for each stage of a pipelined model and develops