Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints
Abstract
We propose ERA, a new paradigm for entropy-constrained policy via output activation. It guarantees minimum sampling entropy by transforming the outputs of the last layer. Our approach demonstrates broad effectiveness across different domains: 1) for large language models(LLMs), boosting the average score across six benchmarks for Qwen2.5-Math-7B by 11.6%; 2) for continuous control reinforcement learning agents, improving performance by more than 30% over strong baselines such as SAC on the challenging HumanoidBench; 3) for image classification, enhancing ImageNet top-1 accuracy by 0.69% for ResNet-50. These gains are achieved with a computational overhead of less than 7%. Our work validates output activation as a powerful tool for entropy control, opening a new direction for designing simpler and more robust algorithms. Code available at: https://nothingbutbut.github.io/era
Cite
Text
Kang et al. "Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints." International Conference on Learning Representations, 2026.Markdown
[Kang et al. "Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/kang2026iclr-entropy/)BibTeX
@inproceedings{kang2026iclr-entropy,
title = {{Entropy Regularizing Activation: Boosting Continuous Control, Large Language Models, and Image Classification with Activation as Entropy Constraints}},
author = {Kang, Zilin and Liao, Chonghua and Xu, Tingqiang and Xu, Huazhe},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/kang2026iclr-entropy/}
}