Video Summarization via Multi-View Representative Selection

Abstract

Video contents are inherently heterogeneous. To exploit different feature modalities in a diverse video collection for video summarization, we propose to formulate the task as a multi-view representative selection problem. The goal is to select visual elements that are representative of a video consistently across different views (i.e., feature modalities). We present in this paper the multi-view sparse dictionary selection with centroid co-regularization (MSDS-CC) method, which optimizes the representative selection in each view, and enforces that the view-specific selections to be similar by regularizing them towards a consensus selection. The problem can be efficiently solved by an alternating minimizing optimization with the fast iterative shrinkage thresholding algorithm (FISTA). We also show how the formulation can be applied to category-specific video summarization by incorporating visual co-occurrence priors. Experiments on benchmark video datasets validate the effectiveness of the proposed approach in comparison with other video summarization methods and representative selection methods.