AetherCode: Evaluating LLMs’ Ability to Win in Premier Programming Competitions
Abstract
Competitive programming has emerged as a critical benchmark for evaluating the reasoning and coding capabilities of Large Language Models (LLMs). Despite impressive progress on existing benchmarks, we argue that current evaluations overstate model proficiency, masking a substantial gap between LLMs and elite human programmers. This gap arises from two key limitations: insufficient difficulty and scope of benchmark problems, and evaluation bias from low-quality test cases. To address these shortcomings, we present **AetherCode**, a new benchmark that draws problems from premier programming competitions such as IOI and ICPC, offering broader coverage and higher difficulty. AetherCode further incorporates comprehensive, expert-validated test suites built through a hybrid of automated generation and human curation, ensuring rigorous and reliable assessment. By combining challenging problem design with robust evaluation, AetherCode provides a more faithful measure of LLM capabilities and sets a new standard for future research in code reasoning.
Cite
Text
Wang et al. "AetherCode: Evaluating LLMs’ Ability to Win in Premier Programming Competitions." International Conference on Learning Representations, 2026.Markdown
[Wang et al. "AetherCode: Evaluating LLMs’ Ability to Win in Premier Programming Competitions." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/wang2026iclr-aethercode/)BibTeX
@inproceedings{wang2026iclr-aethercode,
title = {{AetherCode: Evaluating LLMs’ Ability to Win in Premier Programming Competitions}},
author = {Wang, Zihan and Chen, Jiaze and Liu, Zhicheng and Pan, Haojie and Mak, Markus and Du, Yidi and Moon, Geonsik and Tua, Aaron and Peng, Kunshuo and Lu, Jiayi and Zou, Boqian and Ran, Chenyang and GuangTian, and Zhu, Shoutai and Yeheng, Duan and Kang, Zhenghui and Lin, Zhenxing and Lishangshu, and Luo, Qiang and Long, Qingshen and Chen, Zhiyong and Xiao, Yihan and Wu, Yurong and Zan, Daoguang and Wang, Mingxuan and Ding, Ming},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/wang2026iclr-aethercode/}
}