Multi-Label Image Classification with Multi-Scale Global-Local Semantic Graph Network

Abstract

With the development of deep learning techniques, multi-label image classification tasks have achieved good performance. Recently, graph convolutional network has been proved to be an effective way to explore the labels dependencies. However, due to the complexity of label semantic relations, the static dependencies obtained by existing methods cannot consider the overall characteristics of an image and accurately locate the target region. Therefore, we propose the Multi-scale Global-local Semantic Graph Network (MGSGN) for multi-label image classification, which mainly includes three important parts. First, the multi-scale feature reconstruction aggregates complementary information at different levels in CNN through cross-layer attention, which can effectively identify target categories of different sizes. We then design a channel dual-branch cross-attention module to explore the correlation between global information and local features in multi-scale features, which using the way of adaptive cross-fusion to locate the target area more accurately. Moreover, we propose the multi-perspective weighted cosine measure in multi-perspective dynamic semantic representation module to construct content-based label dependencies for each image to dynamically construct a semantic relationship graph. Extensive experiments on the two public datasets have verified that the classification performance of our model is better than many state-of-the-art methods.