The Bias-Variance Tradeoff and the Randomized GACV
Abstract
We propose a new in-sample cross validation based method (randomized GACV) for choosing smoothing or bandwidth parameters that govern the bias-variance or fit-complexity tradeoff in 'soft' classification. Soft clas(cid:173) sification refers to a learning procedure which estimates the probability that an example with a given attribute vector is in class 1 vs class O. The target for optimizing the the tradeoff is the Kullback-Liebler distance between the estimated probability distribution and the 'true' probabil(cid:173) ity distribution, representing knowledge of an infinite population. The method uses a randomized estimate of the trace of a Hessian and mimics cross validation at the cost of a single relearning with perturbed outcome data.
Cite
Text
Wahba et al. "The Bias-Variance Tradeoff and the Randomized GACV." Neural Information Processing Systems, 1998.Markdown
[Wahba et al. "The Bias-Variance Tradeoff and the Randomized GACV." Neural Information Processing Systems, 1998.](https://mlanthology.org/neurips/1998/wahba1998neurips-biasvariance/)BibTeX
@inproceedings{wahba1998neurips-biasvariance,
title = {{The Bias-Variance Tradeoff and the Randomized GACV}},
author = {Wahba, Grace and Lin, Xiwu and Gao, Fangyu and Xiang, Dong and Klein, Ronald and Klein, Barbara},
booktitle = {Neural Information Processing Systems},
year = {1998},
pages = {620-626},
url = {https://mlanthology.org/neurips/1998/wahba1998neurips-biasvariance/}
}