Position: Principles of Animal Cognition to Improve LLM Evaluations
Abstract
It has become increasingly challenging to understand and evaluate LLM capabilities as these models exhibit a broader range of behaviors. In this position paper, we argue that LLM researchers should draw on the lessons from another field which has developed a rich set of experimental paradigms and design practices for probing the behavior of complex intelligent systems: animal cognition. We present five core principles of evaluation drawn from animal cognition research, and explain how they provide invaluable guidance for understanding LLM capabilities and behavior. We ground these principles in an empirical case study, and show how they can already provide a richer picture of one particular reasoning capability: transitive inference.
Cite
Text
Rane et al. "Position: Principles of Animal Cognition to Improve LLM Evaluations." Proceedings of the 42nd International Conference on Machine Learning, 2025.Markdown
[Rane et al. "Position: Principles of Animal Cognition to Improve LLM Evaluations." Proceedings of the 42nd International Conference on Machine Learning, 2025.](https://mlanthology.org/icml/2025/rane2025icml-position/)BibTeX
@inproceedings{rane2025icml-position,
title = {{Position: Principles of Animal Cognition to Improve LLM Evaluations}},
author = {Rane, Sunayana and Kirkman, Cyrus F. and Todd, Graham and Royka, Amanda and Law, Ryan M.C. and Cartmill, Erica and Foster, Jacob Gates},
booktitle = {Proceedings of the 42nd International Conference on Machine Learning},
year = {2025},
pages = {82051-82061},
volume = {267},
url = {https://mlanthology.org/icml/2025/rane2025icml-position/}
}