Effective Scene Graph Generation by Statistical Relation Distillation

Abstract

Annotating scene graphs for images is a time-consuming task resulting in many instances of missing relations within existing datasets. In this paper we introduce the Statistical Relation Distillation (SRD) method to enhance scenegraph datasets. SRD leverages human-annotated relations alongside object-to-object and predicate-to-predicate similarities to reinforce the existence likelihood of scene graph relations. Moreover SRD can augment relational frequency using relations of non-selected object and predicate categories that are usually omitted by scene graph generation (SGG) tasks. The output from SRD derives the prior probability which is combined with model-predicted probabilities to annotate missing relations in training images and sub-sequently re-train SGG models on the augmented dataset. We evaluate our proposed method on Visual Genome and GQA-200 datasets. Experimental results show that training on the augmented dataset enhances the performance of prominent scene-graph generation models. The implementation code is at https://github.com/LUNAProject22/SRD.