Machine Learning and Information Retrieval (Abstract)

Abstract

This chapter provides an overview of machine learning and information retrieval (IR). The basic processes of IR include query representation, object representation, and retrieval. A number of models of these processes have been developed and used as the basis of system implementations. Examples are the vector space model and probabilistic models for retrieval, and the 2-Poisson model for indexing. Given an understanding of these processes and how they are used in practice, the opportunities for using learning techniques become clearer. The learning techniques are interpreted broadly to include Markov models, clustering, regression, and other techniques from pattern recognition, and more traditional approaches such as neural networks, genetic algorithms, and decision trees. Each of these techniques has been applied to some aspect of IR with varying degrees of success. Examples include stochastic part of speech taggers, relevance feedback based on neural networks, learning document representations using genetic algorithms, learning probability estimation functions, and clustering used to transform the representation space.