Selecting Auxiliary Data Using Knowledge Graphs for Image Classification with Limited Labels

Abstract

In this paper, we propose a learning algorithm for training deep neural networks when there is not sufficient labeled data. To improve the generalization capabilities of the deep model, we adopt a learning scheme to train two related tasks simultaneously. One is the original task (target), and the other is an auxiliary task (source). In order to create a related auxiliary task, we leverage an available knowledge graph to query for semantically related concepts that are grounded in labeled images; hence we call our method KGAuxLearn. We jointly train the target and source tasks in a multi-task architecture. We evaluate our method on two fine-grained visual categorization benchmarks: Oxford Flowers 102 and CUB-200-2011. Our experiments demonstrate that the error rate reduced by at least 2.1% over finetuning for both datasets. We also improve the error rate by 1.36% and 2.93% over using randomly selected concepts as an auxiliary task for Oxford Flowers 102 and CUB-2002011, respectively. In addition, comparing our method with auxiliary data selection methods that do not use a knowledge graph, the error rate improves by 0.69% and 2.57% on Oxford Flowers 102 and CUB-200-2011, respectively.