Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation
Abstract
Large language models (LLMs) often experience language confusion, which is the unintended mixing of languages during text generation. Current solutions to this problem either necessitate model retraining or cannot differentiate between harmful confusion and acceptable code-switching. This paper introduces the \textbf{Language Confusion Gate} (LCG), a lightweight, plug-in solution that filters tokens during decoding without altering the base LLM. The LCG is trained using norm-adjusted self-distillation to predict appropriate language families and apply masking only when needed. Our method is based on the findings that language confusion is infrequent, correct-language tokens are usually among the top predictions, and output token embedding norms are larger for high-resource languages, which biases sampling. When evaluated across various models, including Qwen3, GPT-OSS, Gemma3, Llama3.1, LCG decreases language confusion significantly—often by an order of magnitude—without negatively impacting task performance.
Cite
Text
Zhang et al. "Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation." International Conference on Learning Representations, 2026.Markdown
[Zhang et al. "Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/zhang2026iclr-language/)BibTeX
@inproceedings{zhang2026iclr-language,
title = {{Language Confusion Gate: Language-Aware Decoding Through Model Self-Distillation}},
author = {Zhang, Collin and Huang, Fei and Yuan, Chenhan and Lin, Junyang},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/zhang2026iclr-language/}
}