Learning Complementary Knowledge via Trusted Multi-View Space Decomposition for Self-Supervised Contrastive Learning

Abstract

The contrastive learning paradigm achieves the state-of-the-art in the field of self-supervised learning, which is based on a foundational assumption that the discriminative information is mostly contained in the shared information among views. Yet, we argue that view-shared information alone is not sufficient for specific downstream tasks and certain views often contain complementary knowledge. In this paper, we propose a novel trusted multi-view decoupled contrast method to incorporate such complementary knowledge for self-supervised learning. Instead of only learning in a common space, our method additionally maps the representations into several decoupled spaces. Each decoupled space is associated with a particular view by assuming that the view does not contain view-specific discriminative information, so it is treated as the negative for applying contrastive learning in this space. However, this assumption may not hold for all decoupled spaces. We measure the trustworthiness that the assumption is valid for each space by a Dirichlet distribution-based approach. To tackle the issue that contrastive learning in different spaces affects the optimization of each other, we propose topology-preserving compactness regularization, which enables our method to learn local compact representations robustly against interference. Empirically, our method outperforms state-of-the-art methods in benchmark downstream tasks.