A Unified View of Forward and Backward Losses for Learning from Weak Labels

Abstract

Training multiclass classifiers on weakly labeled datasets, where labels provide only partial or noisy information about the true class, poses a significant challenge in machine learning. To address various forms of label corruption, including noisy, complementary, supplementary, or partial labels, as well as positive-unlabeled data, forward and backward correction losses have been widely employed. Adopting a general formulation that encompasses all these types of label corruption, we introduce a new family of loss functions, termed forward-backward losses, which generalizes both forward and backward correction. We analyze the theoretical properties of this family, providing sufficient conditions under which these losses are proper, ranking-calibrated, classification-calibrated, convex, or lower-bounded. This unified view will be useful to show, through theoretical analysis and experiments, that proper forward losses consistently outperform other forward-backward losses in terms of robustness and accuracy. However, the optimal choice of loss for ranking- and classification-calibrated settings remains an open question. Our work provides a comprehensive framework for weak label learning, offering new directions to develop more robust and effective algorithms.