Actor-Transformers for Group Activity Recognition
Abstract
This paper strives to recognize individual actions and group activities from videos. While existing solutions for this challenging problem explicitly model spatial and temporal relationships based on location of individual actors, we propose an actor-transformer model able to learn and selectively extract information relevant for group activity recognition. We feed the transformer with rich actor-specific static and dynamic representations expressed by features from a 2D pose network and 3D CNN, respectively. We empirically study different ways to combine these representations and show their complementary benefits. Experiments show what is important to transform and how it should be transformed. What is more, actor-transformers achieve state-of-the-art results on two publicly available benchmarks for group activity recognition, outperforming the previous best published results by a considerable margin
Cite
Text
Gavrilyuk et al. "Actor-Transformers for Group Activity Recognition." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. doi:10.1109/CVPR42600.2020.00092Markdown
[Gavrilyuk et al. "Actor-Transformers for Group Activity Recognition." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.](https://mlanthology.org/cvpr/2020/gavrilyuk2020cvpr-actortransformers/) doi:10.1109/CVPR42600.2020.00092BibTeX
@inproceedings{gavrilyuk2020cvpr-actortransformers,
title = {{Actor-Transformers for Group Activity Recognition}},
author = {Gavrilyuk, Kirill and Sanford, Ryan and Javan, Mehrsan and Snoek, Cees G. M.},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2020},
doi = {10.1109/CVPR42600.2020.00092},
url = {https://mlanthology.org/cvpr/2020/gavrilyuk2020cvpr-actortransformers/}
}