Looking at Deep Learning Phenomena Through a Telescoping Lens
Abstract
Deep learning sometimes appears to work in unexpected ways. In pursuit of deeper understanding of its surprising behaviors, we investigate the utility of a tractable and accurate model of a neural network consisting of a sequence of first-order approximations _telescoping_ out into a single empirically operational tool for practical analysis. We illustrate how it can be applied to derive new empirical insights on a diverse range of prominent phenomena in the literature -- including double descent, grokking, and the challenges of applying deep learning on tabular data.
Cite
Text
Jeffares et al. "Looking at Deep Learning Phenomena Through a Telescoping Lens." ICML 2024 Workshops: HiLD, 2024.Markdown
[Jeffares et al. "Looking at Deep Learning Phenomena Through a Telescoping Lens." ICML 2024 Workshops: HiLD, 2024.](https://mlanthology.org/icmlw/2024/jeffares2024icmlw-looking/)BibTeX
@inproceedings{jeffares2024icmlw-looking,
title = {{Looking at Deep Learning Phenomena Through a Telescoping Lens}},
author = {Jeffares, Alan and Curth, Alicia and van der Schaar, Mihaela},
booktitle = {ICML 2024 Workshops: HiLD},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/jeffares2024icmlw-looking/}
}