An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models
Abstract
We consider the task of tuning hyperparameters in SVM models based on minimizing a smooth performance validation function, e.g., smoothed k-fold crossvalidation error, using non-linear optimization techniques. The key computation in this approach is that of the gradient of the validation function with respect to hyperparameters. We show that for large-scale problems involving a wide choice of kernel-based models and validation functions, this computation can be very efficiently done; often within just a fraction of the training time. Empirical results show that a near-optimal set of hyperparameters can be identified by our approach with very few training rounds and gradient computations. .
Cite
Text
Keerthi et al. "An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models." Neural Information Processing Systems, 2006.Markdown
[Keerthi et al. "An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models." Neural Information Processing Systems, 2006.](https://mlanthology.org/neurips/2006/keerthi2006neurips-efficient/)BibTeX
@inproceedings{keerthi2006neurips-efficient,
title = {{An Efficient Method for Gradient-Based Adaptation of Hyperparameters in SVM Models}},
author = {Keerthi, S. S. and Sindhwani, Vikas and Chapelle, Olivier},
booktitle = {Neural Information Processing Systems},
year = {2006},
pages = {673-680},
url = {https://mlanthology.org/neurips/2006/keerthi2006neurips-efficient/}
}