Kernel Methods for Missing Variables

Abstract

We present methods for dealing with missing variables in the context of Gaussian Processes and Support Vector Machines. This solves an important problem which has largely been ig-nored by kernel methods: How to systemati-cally deal with incomplete data? Our method can also be applied to problems with partially observed labels as well as to the transductive setting where we view the labels as missing data. Our approach relies on casting kernel meth-ods as an estimation problem in exponen-tial families. Hence, estimation with miss-ing variables becomes a problem of comput-ing marginal distributions, and finding effi-cient optimization methods. To that extent we propose an optimization scheme which ex-tends the Concave Convex Procedure (CCP) of Yuille and Rangarajan, and present a simplified and intuitive proof of its conver-gence. We show how our algorithm can be specialized to various cases in order to effi-ciently solve the optimization problems that arise. Encouraging preliminary experimen-tal results on the USPS dataset are also pre-sented. 1