On Separability of Loss Functions, and Revisiting Discriminative vs Generative Models
Abstract
We revisit the classical analysis of generative vs discriminative models for general exponential families, and high-dimensional settings. Towards this, we develop novel technical machinery, including a notion of separability of general loss functions, which allow us to provide a general framework to obtain l∞ convergence rates for general M-estimators. We use this machinery to analyze l∞ and l2 convergence rates of generative and discriminative models, and provide insights into their nuanced behaviors in high-dimensions. Our results are also applicable to differential parameter estimation, where the quantity of interest is the difference between generative model parameters.
Cite
Text
Prasad et al. "On Separability of Loss Functions, and Revisiting Discriminative vs Generative Models." Neural Information Processing Systems, 2017.Markdown
[Prasad et al. "On Separability of Loss Functions, and Revisiting Discriminative vs Generative Models." Neural Information Processing Systems, 2017.](https://mlanthology.org/neurips/2017/prasad2017neurips-separability/)BibTeX
@inproceedings{prasad2017neurips-separability,
title = {{On Separability of Loss Functions, and Revisiting Discriminative vs Generative Models}},
author = {Prasad, Adarsh and Niculescu-Mizil, Alexandru and Ravikumar, Pradeep K},
booktitle = {Neural Information Processing Systems},
year = {2017},
pages = {7050-7059},
url = {https://mlanthology.org/neurips/2017/prasad2017neurips-separability/}
}