Uncertainty-Based Forgetting Mitigation for Generalized Few-Shot Object Detection

Abstract

Generalized Few-Shot Object Detection (G-FSOD) seeks to jointly detect base classes with abundant data and novel classes with limited data. Due to data scarcity, predictive uncertainties are more pronounced in G-FSOD than in conventional object detection. Unaccounting for these uncertainties leads to degraded overall detection performance and forgetting the base classes. However, previous G-FSOD works have not exploited these uncertainties. Upon examining the basic two-stage G-FSOD framework, which includes a Region Proposal Network (RPN) and a subsequent R-CNN, we observe that a straightforward integration of uncertainty estimation leads to detrimental performance. To this end, we first increase the model capacity by increasing the depth of the RPN and cascading multiple R-CNNs in an end-to-end manner. Next, we interleave the stages with uncertainty estimation and attention blocks. The aim is to progressively refine the proposals by exploiting the estimated uncertainties while attending to the discriminative features through the attention mechanism. Extensive experiments on the well-established G-FSOD benchmarks, MS-COCO and PASCAL-VOC, show that our proposed method sets a new G-FSOD standard.