Simple and Efficient Multiple Kernel Learning by Group Lasso

Abstract

We consider the problem of how to improve the efficiency of Multiple Kernel Learning (MKL). MKL is often regarded as a convex-concave optimization problem, which is convex on the kernel weights and concave on the SVM dual variables. In literature, the alternating approach is widely observed: (1) the minimization of the kernel weights is solved by complicated techniques, such as Semi-infinite Linear Programming, Gradient Descent, or Level method; (2) the maximization of SVM dual variables can be solved by standard SVM solvers. However, the minimization over kernel weights in these methods is usually dependent on its solving techniques or commercial softwares, which therefore limits the efficiency and applicability. In this paper, we formulate a closed-form solution for the optimization of kernel weights based on the equivalence between group-lasso and MKL. Although this equivalence is not our invention, our derived variant equivalence not only lead to an efficient algorithm for MKL, but also generalize this equivalence to that between Lp-MKL (p >=1 and denoting the Lp-norm of kernel weights) and group lasso, which does not appear in literature. Therefore, our proposed algorithm provides a unified solution for the whole family of Lp-MKL models. Experiments on multiple data sets show the promising performance of the proposed technique compared with other competitive methods.