A Consistent Estimator of the Expected Gradient Outerproduct
Abstract
In high-dimensional classification or regression problems, the expected gradient outerproduct (EGOP) of the unknown regression function f, namely Ex (∇ f(X) · ∇f(X)⊤), is known to recover those directions v ∈ ℝd most relevant to predicting the output Y. However, just as in gradient estimation, optimal estimators of the EGOP can be expensive in practice. We show that a simple rough estimator, much cheaper in practice, suffices to obtain significant improvements on real-world nonparametric classification and regression tasks. Furthermore, we prove that, despite its simplicity, this rough estimator remains statistically consistent under mild conditions.
Cite
Text
Trivedi et al. "A Consistent Estimator of the Expected Gradient Outerproduct." Conference on Uncertainty in Artificial Intelligence, 2014.Markdown
[Trivedi et al. "A Consistent Estimator of the Expected Gradient Outerproduct." Conference on Uncertainty in Artificial Intelligence, 2014.](https://mlanthology.org/uai/2014/trivedi2014uai-consistent/)BibTeX
@inproceedings{trivedi2014uai-consistent,
title = {{A Consistent Estimator of the Expected Gradient Outerproduct}},
author = {Trivedi, Shubhendu and Wang, Jialei and Kpotufe, Samory and Shakhnarovich, Gregory},
booktitle = {Conference on Uncertainty in Artificial Intelligence},
year = {2014},
pages = {819-828},
url = {https://mlanthology.org/uai/2014/trivedi2014uai-consistent/}
}