Gradient LASSO for Feature Selection

Abstract

LASSO (Least Absolute Shrinkage and Selection Operator) is a useful tool toachieve the shrinkage and variable selection simultaneously. Since LASSO uses the $L_1$ penalty, the optimization should rely on the quadratic program (QP) or generalnon-linear program which is known to be computational intensive. In this paper, we propose agradient descent algorithm for LASSO. Even though the final result is slightly less accurate, the proposed algorithmis computationally simpler than QP or non-linear program, and so can be applied to large size problems. We provide the convergence rate of the algorithm, and illustrate it withsimulated models as well as real data sets.