Explicit Regularisation, Sharpness and Calibration

Abstract

We probe the relation between flatness, generalisation and calibration in neural networks, using explicit regularisation as a control variable. Our findings indicate that the range of flatness metrics surveyed fail to positively correlate with variation in generalisation or calibration. In fact, the correlation is often opposite to what has been hypothesized or claimed in prior work, with calibrated models typically existing at sharper minima compared to relative baselines, this relation exists across model classes and dataset complexities.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Mason-Williams et al. "Explicit Regularisation, Sharpness and Calibration." NeurIPS 2024 Workshops: SciForDL, 2024.

Markdown

[Mason-Williams et al. "Explicit Regularisation, Sharpness and Calibration." NeurIPS 2024 Workshops: SciForDL, 2024.](https://mlanthology.org/neuripsw/2024/masonwilliams2024neuripsw-explicit/)

BibTeX

@inproceedings{masonwilliams2024neuripsw-explicit,
  title     = {{Explicit Regularisation, Sharpness and Calibration}},
  author    = {Mason-Williams, Israel and Ekholm, Fredrik and Huszár, Ferenc},
  booktitle = {NeurIPS 2024 Workshops: SciForDL},
  year      = {2024},
  url       = {https://mlanthology.org/neuripsw/2024/masonwilliams2024neuripsw-explicit/}
}