Efficient Retrieval from Large-Scale Egocentric Visual Data Using a Sparse Graph Representation

Abstract

We propose representing one's visual experiences (captured as a series of ego-centric videos) as a sparse-graph, where each node is an individual frame in the video, and nodes are connected if there exists a geometric transform between them. Such a graph is massive and contains millions of edges. Autobiographical egocentric visual data are highly redundant, and we show how the graph representation and graph clustering can be used to exploit redundancy in the data. We show that popular global clustering methods like spectral clustering and multi-level graph partitioning perform poorly for clustering egocentric visual data. We propose using local density clustering algorithms for clustering the data, and provide detailed qualitative and quantitative comparisons between the two approaches. The graph-representation and clustering are used to aggressively prune the database. By retaining only representative nodes from dense sub graphs, we achieve 90% of peak recall by retaining only 1% of data, with a significant 18% improvement in absolute recall over naive uniform subsampling of the egocentric video data.