ML Anthology

Efficient Fine-Tuning Large Language Models for Knowledge-Aware Response Planning

Minh Nguyen, K. C. Kishan, Toàn Quoc Nguyên, Ankit Chadha, Thuy Vu
ECML-PKDD 2023 pp. 593-611
doi:10.1007/978-3-031-43415-0_35 /ecmlpkdd/2023/nguyen2023ecmlpkdd-efficient/

Abstract

Large Language Models (LLMs) have shown impressive emergent language capabilities, especially in applications with high ambiguity, such as language reasoning and knowledge consolidation. However, previous work explores the use of LLMs for acquiring information using either parametric or external knowledge, which might lead to serious issues such as hallucination. Toward solving these issues, we present a novel approach of knowledge-aware response planning (KARP) and propose a novel framework that employs (i) a knowledge retriever to obtain relevant information from web documents or databases for a given user query, and (ii) a robust fine-tuning strategy for LLMs to exploit the retrieved external knowledge for planning a final response. Experimental results show that our proposed framework can provide natural, concise answers for open-domain questions with high accuracy.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Nguyen et al. "Efficient Fine-Tuning Large Language Models for Knowledge-Aware Response Planning." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023. doi:10.1007/978-3-031-43415-0_35

Markdown

[Nguyen et al. "Efficient Fine-Tuning Large Language Models for Knowledge-Aware Response Planning." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2023.](https://mlanthology.org/ecmlpkdd/2023/nguyen2023ecmlpkdd-efficient/) doi:10.1007/978-3-031-43415-0_35

BibTeX

@inproceedings{nguyen2023ecmlpkdd-efficient,
  title     = {{Efficient Fine-Tuning Large Language Models for Knowledge-Aware Response Planning}},
  author    = {Nguyen, Minh and Kishan, K. C. and Nguyên, Toàn Quoc and Chadha, Ankit and Vu, Thuy},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2023},
  pages     = {593-611},
  doi       = {10.1007/978-3-031-43415-0_35},
  url       = {https://mlanthology.org/ecmlpkdd/2023/nguyen2023ecmlpkdd-efficient/}
}