Natural Language Generation with Expert Standards

Abstract

Standards, or expert-defined preferences, are documented guidelines describing strict specifications for text-based content such as books, manuals, and reports. These guidelines are curated, defined, and continuously improved by domain experts in various fields, such as education, policy, and healthcare, and are used for maintaining quality. In my dissertation, I focus on evaluating and teaching large language models (LLMs) to capture standards to improve generation quality across diverse language generation tasks. I draw motivation from my preliminary published works, where I explored how open and commercial LLMs can learn complex constraints from standards in education and language assessment to produce classroom-ready narrative content. In this proposal, I also discuss the technical novelty, impact, and target contributions and highlight how this line of work can be scaled and generalized for other domains where standards are also used as a reference of quality.