Sparse Additive Machine

Abstract

We develop a high dimensional nonparametric classification method named sparse additive machine (SAM), which can be viewed as a functional version of support vector machines (SVM) combined with sparse additive modeling. SAM is related to multiple kernel learning (MKL), but is computationally more efficient and amenable to theoretical analysis. In terms of computation, we develop an efficient accelerated proximal gradient descent algorithm which is also scalable to large data sets with a provable O(1/k^2) convergence rate and k is the number of iterations. In terms of theory, we provide the oracle properties of SAM under asymptotic frameworks. Empirical results on3 both synthetic and real data are reported to back up our theory.