Learning to Count from Pseudo-Labeled Segmentation

Abstract

Class-agnostic counting (CAC) has numerous potential applications across various domains. The goal is to count objects of an arbitrary category during testing based on only a few annotated exemplars. However existing methods often count all objects in the image including those from different categories than the exemplars. To address this issue we propose localizing the area containing the objects of interest via an exemplar-based segmentation model before counting them. To train this model we propose a novel method to obtain pseudo-labeled segmentation masks. Specifically we use an unsupervised image clustering method to generate a set of candidate pseudo object masks from which we select the optimal one using a pre-trained CAC model. We show that the trained segmentation model can effectively localize objects of interest based on the exemplars and prevent the model from counting everything. To properly evaluate the performance of CAC methods in real-world scenarios we introduce two new benchmarks: a synthetic test set and a new test set of real images containing countable objects from multiple classes. Our proposed method shows a significant advantage over previous CAC methods on these two benchmarks.