Level Selector Network for Optimizing Accuracy-Specificity Trade-Offs

Abstract

With the increase in visual categories that become more and more fine-granular, maintaining high accuracy is a challenge. As the visual world can be organized in a semantic hierarchy, which is usually in form of a directed acyclic graph of many levels of abstraction, a classifier should be able to select an appropriate level trading off specificity for accuracy in case of uncertainty. In this work, we study the problem of finding accuracy vs. specificity trade-offs. To this end, we propose a Level Selector network, which selects the class granularity for the class prediction for an image or video, and a self-supervision based training strategy to train the Level Selector network. We show as part of the empirical evaluation, that our approach achieves superior results compared to the current state of the art on large-scale image and video datasets.