SPARTAN: Self-Supervised Spatiotemporal Transformers Approach to Group Activity Recognition

Naga Venkata Sai Raviteja Chappa, Pha A. Nguyen, Alexander H. Nelson, Han-Seok Seo, Xin Li, Page Daniel Dobbs, Khoa Luu

CVPRW 2023 pp. 5158-5168

doi:10.1109/CVPRW59228.2023.00544 /cvprw/2023/chappa2023cvprw-spartan/

Abstract

In this paper, we propose a new, simple, and effective Self-supervised Spatio-temporal Transformers (SPARTAN) approach to Group Activity Recognition (GAR) using unlabeled video data. Given a video, we create local and global Spatio-temporal views with varying spatial patch sizes and frame rates. The proposed self-supervised objective aims to match the features of these contrasting views representing the same video to be consistent with the variations in spatiotemporal domains. To the best of our knowledge, the proposed mechanism is one of the first works to alleviate the weakly supervised setting of GAR using the encoders in video transformers. Furthermore, using the advantage of transformer models, our proposed approach supports long-term relationship modeling along spatio-temporal dimensions. The proposed SPARTAN approach performs well on two group activity recognition benchmarks, including NBA and Volleyball datasets, by surpassing the state-of-the-art results by a significant margin in terms of MCA and MPCA metrics1.

PDF CVPRW Semantic Scholar

Cite

Text

Chappa et al. "SPARTAN: Self-Supervised Spatiotemporal Transformers Approach to Group Activity Recognition." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023. doi:10.1109/CVPRW59228.2023.00544

Markdown

[Chappa et al. "SPARTAN: Self-Supervised Spatiotemporal Transformers Approach to Group Activity Recognition." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2023.](https://mlanthology.org/cvprw/2023/chappa2023cvprw-spartan/) doi:10.1109/CVPRW59228.2023.00544

BibTeX

@inproceedings{chappa2023cvprw-spartan,
  title     = {{SPARTAN: Self-Supervised Spatiotemporal Transformers Approach to Group Activity Recognition}},
  author    = {Chappa, Naga Venkata Sai Raviteja and Nguyen, Pha A. and Nelson, Alexander H. and Seo, Han-Seok and Li, Xin and Dobbs, Page Daniel and Luu, Khoa},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2023},
  pages     = {5158-5168},
  doi       = {10.1109/CVPRW59228.2023.00544},
  url       = {https://mlanthology.org/cvprw/2023/chappa2023cvprw-spartan/}
}