Extracting Relevant Snippets for Web Navigation

Abstract

Search engines present fix-length passages from documents ranked by relevance against the query. In this paper, we present and compare novel, language-model based methods for extracting variable length document snippets by real-time processing of documents using the query issued by the user. With this extra level of information, the returned snippets are considerably more informative. Unlike previous work on passage retrieval which relies on searching relevant segments for filtering of preoccupied passages, we focus on query-informed segmentation to extract context-aware relevant snippets with variable length. In particular, we show that, when informed through an appropriate relevance language model, curvature analysis and Hidden Markov model (HMM) based content segmentation techniques can facilitate to extract relevant document snippets.