Causal Retrieval with Semantic Consideration

Abstract

Recent rapid progress in large language models (LLMs) leads to high performance conversational AI system. To extend the the system to expert fields, such as biomedical or legal domains, it becomes standard to combine LLMs with information retrieval (IR) system and generate answer based on retrieved information (documents) for given queries. Thus, it is essential that the IR system should ''understand'' various intents included in the queries including but not limited to semantic similarity, causal relationship etc. However, existing IR systems primarily focuses on retrieving related information based only on semtnic similarities between sentences or documents. Here, we develop CAWAI that can understand causal relation between queries and documents . By training dense retriever with dual constraints, causal loss and semantic loss, CAWAI shows strong generalization capability achieving up to +7.8\% Hit@1 compared to DPR baseline in causal retrieval task where a target sentence is buried in 20m Wikipedia sentences.