SCoRe: Pre-Training for Context Representation in Conversational Semantic Parsing

Abstract

Conversational Semantic Parsing (CSP) is the task of converting a sequence of natural language queries to formal queries (e.g., SQL, SPARQL) to be executed against a structured ontology (e.g. databases, KBs). A CSP system needs to model the alignment between the unstructured language utterance and the structured ontology in the context of multi-turn dialog dynamics. Pre-trained language models have limited ability to represent NL references to structural data. We present SCoRe, a new pre-training approach for CSP tasks designed to induce representations that capture the alignment between the conversational flow and the structural context. By combining SCoRe with strong base systems on four different tasks (SParC, CoSQL, MWoZ, and SQA), we improve the performance over all baselines by a significant margin and achieve state-of-the-art results on three of them.