Semantic Video-to-Video Search Using Sub-Graph Grouping and Matching

Abstract

We propose a novel video event retrieval algorithm given a video query containing grouped events from large scale video database. Rather than looking for similar scenes using visual features as conventional image retrieval algorithms do, we search for the similar semantic events (e.g. finding a video such that a person parks a vehicle and meets with other person and exchanges a bag). Videos are analyzed semantically and represented by a graphical structure. Now the problem is to match the graph with other graphs of events in the database. Since the query video may include noisy activities or some event may not be detected by the semantic video analyzer, exact graph matching does not always work. For efficient and effective solution, we introduce a novel sub graph indexing and matching scheme. Sub graphs are grouped and their importance is further learned over video by topic learning algorithms. After grouping and indexing sub graphs, the complex graph matching problem becomes simple vector comparison in reduced dimension. The performances are extensively evaluated and compared with each approach.