The Landmark Selection Method for Multiple Output Prediction

Abstract

Conditional modeling x → y is a central problem in machine learning. A substantial research effort is devoted to such modeling when x is high dimensional. We consider, instead, the case of a high dimensional y, where x is either low dimensional or high dimensional. Our approach is based on selecting a small subset yL of the dimensions of y, and proceed by modeling (i) x → yL and (ii) yL → y. Composing these two models, we obtain a conditional model x → y that possesses convenient statistical properties. Multilabel classification and multivariate regression experiments on several datasets show that this method outperforms the one vs. all approach as well as several sophisticated multiple output prediction methods.