Thematic Saliency Detection Using Spatial-Temporal Context

Abstract

We propose a new measurement of video saliency termed thematic video saliency}. Video saliency is detected in terms of finding the thematic objects that frequently appear at the salient positions in the video scenes. By representing all image segments in the video as the spatial-temporal context, we build an affinity graph among them, and formulate the thematic object discovery as a novel cohesive sub-graph mining problem. A trust region algorithm is also proposed to solve the challenging optimization problem. Unlike individual image saliency or co-saliency analysis, our proposed video saliency fully incorporates the whole spatial-temporal video context. Experiments on our newly developed eye tracking dataset as well as other two datasets further validate the effectiveness of our method on video saliency detection.