A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos
Abstract
Existing approaches for spatio-temporal action detection in videos are limited by the spatial extent and temporal duration of the actions. In this paper, we present a modular system for spatio-temporal action detection in untrimmed surveillance videos. We propose a two stage approach. The first stage generates dense spatio-temporal proposals using hierarchical clustering and temporal jittering techniques on frame-wise object detections. The second stage is a Temporal Refinement I3D (TRI-3D) network that performs action classification and temporal refinement on the generated proposals. The object detection-based proposal generation step helps in detecting actions occurring in a small spatial region of a video frame, while temporal jittering and refinement helps in detecting actions of variable lengths. Experimental results on an unconstrained surveillance action detection dataset - DIVA - show the effectiveness of our system. For comparison, the performance of our system is also evaluated on the THUMOS'14 temporal action detection dataset.
Cite
Text
Gleason et al. "A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos." IEEE/CVF Winter Conference on Applications of Computer Vision, 2019. doi:10.1109/WACV.2019.00021Markdown
[Gleason et al. "A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos." IEEE/CVF Winter Conference on Applications of Computer Vision, 2019.](https://mlanthology.org/wacv/2019/gleason2019wacv-proposal/) doi:10.1109/WACV.2019.00021BibTeX
@inproceedings{gleason2019wacv-proposal,
title = {{A Proposal-Based Solution to Spatio-Temporal Action Detection in Untrimmed Videos}},
author = {Gleason, Joshua and Ranjan, Rajeev and Schwarcz, Steven and Castillo, Carlos Domingo and Chen, Jun-Cheng and Chellappa, Rama},
booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
year = {2019},
pages = {141-150},
doi = {10.1109/WACV.2019.00021},
url = {https://mlanthology.org/wacv/2019/gleason2019wacv-proposal/}
}