A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning

Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Girshick, Kaiming He

CVPR 2021 pp. 3299-3309

doi:10.1109/CVPR46437.2021.00331 /cvpr/2021/feichtenhofer2021cvpr-largescale/

Abstract

We present a large-scale study on unsupervised spatiotemporal representation learning from videos. With a unified perspective on four recent image-based frameworks, we study a simple objective that can easily generalize all these methods to space-time. Our objective encourages temporally-persistent features in the same video, and in spite of its simplicity, it works surprisingly well across: (i) different unsupervised frameworks, (ii) pre-training datasets, (iii) downstream datasets, and (iv) backbone architectures. We draw a series of intriguing observations from this study, e.g., we discover that encouraging long-spanned persistency can be effective even if the timespan is 60 seconds. In addition to state-of-the-art results in multiple benchmarks, we report a few promising cases in which unsupervised pre-training can outperform its supervised counterpart. Code will be made available at https://github.com/facebookresearch/SlowFast.

PDF CVPR Semantic Scholar

Cite

Text

Feichtenhofer et al. "A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning." Conference on Computer Vision and Pattern Recognition, 2021. doi:10.1109/CVPR46437.2021.00331

Markdown

[Feichtenhofer et al. "A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning." Conference on Computer Vision and Pattern Recognition, 2021.](https://mlanthology.org/cvpr/2021/feichtenhofer2021cvpr-largescale/) doi:10.1109/CVPR46437.2021.00331

BibTeX

@inproceedings{feichtenhofer2021cvpr-largescale,
  title     = {{A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning}},
  author    = {Feichtenhofer, Christoph and Fan, Haoqi and Xiong, Bo and Girshick, Ross and He, Kaiming},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2021},
  pages     = {3299-3309},
  doi       = {10.1109/CVPR46437.2021.00331},
  url       = {https://mlanthology.org/cvpr/2021/feichtenhofer2021cvpr-largescale/}
}