A Quasi-Newton Approach to Non-Smooth Convex Optimization
Abstract
We extend the well-known BFGS quasi-Newton method and its limited-memory variant (LBFGS) to the optimization of nonsmooth convex objectives. This is done in a rigorous fashion by generalizing three components of BFGS to subdifferentials: The local quadratic model, the identification of a descent direction, and the Wolfe line search conditions. We apply the resulting sub(L)BFGS algorithm to L 2 -regularized risk minimization with binary hinge loss, and its direction-finding component to L 1 -regularized risk minimization with logistic loss. In both settings our generic algorithms perform comparable to or better than their counterparts in specialized state-of-the-art solvers.
Cite
Text
Yu et al. "A Quasi-Newton Approach to Non-Smooth Convex Optimization." International Conference on Machine Learning, 2008. doi:10.1145/1390156.1390309Markdown
[Yu et al. "A Quasi-Newton Approach to Non-Smooth Convex Optimization." International Conference on Machine Learning, 2008.](https://mlanthology.org/icml/2008/yu2008icml-quasi/) doi:10.1145/1390156.1390309BibTeX
@inproceedings{yu2008icml-quasi,
title = {{A Quasi-Newton Approach to Non-Smooth Convex Optimization}},
author = {Yu, Jin and Vishwanathan, S. V. N. and Günter, Simon and Schraudolph, Nicol N.},
booktitle = {International Conference on Machine Learning},
year = {2008},
pages = {1216-1223},
doi = {10.1145/1390156.1390309},
url = {https://mlanthology.org/icml/2008/yu2008icml-quasi/}
}