Towards a Theoretical Understanding of Negative Transfer in Collective Matrix Factorization

Abstract

Collective matrix factorization (CMF) is a popular technique to improve the overall factorization quality of multiple matrices presuming they share the same latent factor. However, it suffers from performance degeneration when this assumption fails, an effect called negative transfer (n.t.). Although the effect is widely admitted, its theoretical nature remains a mystery to date. This paper presents a first theoretical understanding of n.t. in theory. Under the statistical mini-max framework, we derive lower bounds for the CMF estimator and gain two insights. First, the n.t. effect can be explained as the rise of a bias term in the standard lower bound, which depends only on the structure of factor space but neither the estimator nor samples. Second, the n.t. effect can be explained as the rise of an d_th-root function on the learning rate, where d is the dimension of a Grassmannian containing the subspaces spanned by latent factors. These discoveries are also supported in simulation, and suggest n.t. may be more effectively addressed via model construction other than model selection.