Increasing Biases Can Be More Efficient than Increasing Weights
Abstract
We introduce a novel computational unit for neural networks that features multiple biases, challenging the traditional perceptron structure. This unit emphasizes the importance of preserving uncorrupted information as it is passed from one unit to the next, applying activation functions later in the process with specialized biases for each unit. Through both empirical and theoretical analyses, we show that by focusing on increasing biases rather than weights, there is potential for significant enhancement in a neural network model's performance. This approach offers an alternative perspective on optimizing information flow within neural networks. Commented source code at https://github.com/CuriosAI/dac-dev.
Cite
Text
Metta et al. "Increasing Biases Can Be More Efficient than Increasing Weights." Winter Conference on Applications of Computer Vision, 2024.Markdown
[Metta et al. "Increasing Biases Can Be More Efficient than Increasing Weights." Winter Conference on Applications of Computer Vision, 2024.](https://mlanthology.org/wacv/2024/metta2024wacv-increasing/)BibTeX
@inproceedings{metta2024wacv-increasing,
title = {{Increasing Biases Can Be More Efficient than Increasing Weights}},
author = {Metta, Carlo and Fantozzi, Marco and Papini, Andrea and Amato, Gianluca and Bergamaschi, Matteo and Galfrè, Silvia Giulia and Marchetti, Alessandro and Vegliò, Michelangelo and Parton, Maurizio and Morandin, Francesco},
booktitle = {Winter Conference on Applications of Computer Vision},
year = {2024},
pages = {2810-2819},
url = {https://mlanthology.org/wacv/2024/metta2024wacv-increasing/}
}