On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning

Abstract

We derive an analytically tractable theory of SSL landscape and show that it accurately captures an array of collapse phenomena and identifies their causes.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Ziyin et al. "On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning." NeurIPS 2022 Workshops: NeurReps, 2022.

Markdown

[Ziyin et al. "On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning." NeurIPS 2022 Workshops: NeurReps, 2022.](https://mlanthology.org/neuripsw/2022/ziyin2022neuripsw-rotational/)

BibTeX

@inproceedings{ziyin2022neuripsw-rotational,
  title     = {{On Rotational Symmetry in the Loss Landscape of Self-Supervised Learning}},
  author    = {Ziyin, Liu and Lubana, Ekdeep Singh and Ueda, Masahito and Tanaka, Hidenori},
  booktitle = {NeurIPS 2022 Workshops: NeurReps},
  year      = {2022},
  url       = {https://mlanthology.org/neuripsw/2022/ziyin2022neuripsw-rotational/}
}