Oceanic Scene Recognition Using Graph-of-Words (GoW)

Abstract

In this paper, we focus on recognition of oceanic scene images. This task is particularly important for monitoring the oceanic environment with the cameras mounted at different locations. For this purpose, a new image dataset, namely, Flickr Oceanic Scene Dataset (FOSD) 1, is collected. Although it is intuitive to use this dataset to train a Convolutional Neural Network (CNN) from scratch, the relatively limited size of this dataset prevents us from doing so. One option is to encode the visual words learnt from deep convolutional features and it has been shown that these visual words outperform the fully-connected (FC) features extracted using a pre-trained CNN. However, it is commonly known that these word encoders generally do not utilise the spatial layout of words, whereas the spatial information is important to representation of long-range image characteristics. Thus, we propose a new image descriptor: Graph-of-Words (GoW), to capture the higher order spatial relationship between words, simply because graphs are able to encode the complicated spatial layout of node. This descriptor is also fused with three state-of-the-art word encoders to exploit richer characteristics. The GoW descriptor and the fused variants produce promising results in the oceanic and aerial scene recognition tasks. We attribute these results to the fact that the GoW descriptor encodes both the short-range and long-range higher-order spatial relationships between words.