Sycophancy Claims About Language Models: The Missing Human-in-the-Loop
Abstract
Sycophantic response patterns in Large Language Models (LLMs) have been increasingly claimed in the literature. We review methodological challenges in measuring LLM sycophancy and identify five core operationalizations. Despite sycophancy being inherently human-centric, current research does not evaluate human perception. Our analysis highlights the difficulties in distinguishing sycophantic responses from related concepts in AI alignment and offers actionable recommendations for future research.
Cite
Text
Batzner et al. "Sycophancy Claims About Language Models: The Missing Human-in-the-Loop." ICLR 2025 Workshops: Bi-Align, 2025.Markdown
[Batzner et al. "Sycophancy Claims About Language Models: The Missing Human-in-the-Loop." ICLR 2025 Workshops: Bi-Align, 2025.](https://mlanthology.org/iclrw/2025/batzner2025iclrw-sycophancy/)BibTeX
@inproceedings{batzner2025iclrw-sycophancy,
title = {{Sycophancy Claims About Language Models: The Missing Human-in-the-Loop}},
author = {Batzner, Jan and Stocker, Volker and Schmid, Stefan and Kasneci, Gjergji},
booktitle = {ICLR 2025 Workshops: Bi-Align},
year = {2025},
url = {https://mlanthology.org/iclrw/2025/batzner2025iclrw-sycophancy/}
}