Till the Layers Collapse: Compressing a Deep Neural Network Through the Lenses of Batch Normalization Layers
Abstract
Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. In this paper, we introduce a method called Till the Layers Collapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks.
Cite
Text
Liao et al. "Till the Layers Collapse: Compressing a Deep Neural Network Through the Lenses of Batch Normalization Layers." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I18.34058Markdown
[Liao et al. "Till the Layers Collapse: Compressing a Deep Neural Network Through the Lenses of Batch Normalization Layers." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/liao2025aaai-till/) doi:10.1609/AAAI.V39I18.34058BibTeX
@inproceedings{liao2025aaai-till,
title = {{Till the Layers Collapse: Compressing a Deep Neural Network Through the Lenses of Batch Normalization Layers}},
author = {Liao, Zhu and Hezbri, Nour and Quétu, Victor and Nguyen, Van-Tam and Tartaglione, Enzo},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2025},
pages = {18702-18710},
doi = {10.1609/AAAI.V39I18.34058},
url = {https://mlanthology.org/aaai/2025/liao2025aaai-till/}
}