Extensible Prompts for Language Models on Zero-Shot Language Style Customization
Abstract
We propose eXtensible Prompt (X-Prompt) for prompting a large language model (LLM) beyond natural language (NL). X-Prompt instructs an LLM with not only NL but also an extensible vocabulary of imaginary words. Registering new imaginary words allows us to instruct the LLM to comprehend concepts that are difficult to describe with NL words, thereby making a prompt more descriptive. Also, these imaginary words are designed to be out-of-distribution (OOD) robust so that they can be (re)used like NL words in various prompts, distinguishing X-Prompt from soft prompt that is for fitting in-distribution data. We propose context-augmented learning (CAL) to learn imaginary words for general usability, enabling them to work properly in OOD (unseen) prompts. We experiment X-Prompt for zero-shot language style customization as a case study. The promising results of X-Prompt demonstrate its potential to facilitate advanced interaction beyond the natural language interface, bridging the communication gap between humans and LLMs.
Cite
Text
Ge et al. "Extensible Prompts for Language Models on Zero-Shot Language Style Customization." Neural Information Processing Systems, 2023.Markdown
[Ge et al. "Extensible Prompts for Language Models on Zero-Shot Language Style Customization." Neural Information Processing Systems, 2023.](https://mlanthology.org/neurips/2023/ge2023neurips-extensible/)BibTeX
@inproceedings{ge2023neurips-extensible,
title = {{Extensible Prompts for Language Models on Zero-Shot Language Style Customization}},
author = {Ge, Tao and Jing, Hu and Dong, Li and Mao, Shaoguang and Xia, Yan and Wang, Xun and Chen, Si-Qing and Wei, Furu},
booktitle = {Neural Information Processing Systems},
year = {2023},
url = {https://mlanthology.org/neurips/2023/ge2023neurips-extensible/}
}