A Three Sample Hypothesis Test for Evaluating Generative Models
Abstract
Detecting overfitting in generative models is an important challenge in machine learning. In this work, we formalize a form of overfitting that we call {\em{data-copying}} – where the generative model memorizes and outputs training samples or small variations thereof. We provide a three sample test for detecting data-copying that uses the training set, a separate sample from the target distribution, and a generated sample from the model, and study the performance of our test on several canonical models and datasets.
Cite
Text
Meehan et al. "A Three Sample Hypothesis Test for Evaluating Generative Models." Artificial Intelligence and Statistics, 2020.Markdown
[Meehan et al. "A Three Sample Hypothesis Test for Evaluating Generative Models." Artificial Intelligence and Statistics, 2020.](https://mlanthology.org/aistats/2020/meehan2020aistats-three/)BibTeX
@inproceedings{meehan2020aistats-three,
title = {{A Three Sample Hypothesis Test for Evaluating Generative Models}},
author = {Meehan, Casey and Chaudhuri, Kamalika and Dasgupta, Sanjoy},
booktitle = {Artificial Intelligence and Statistics},
year = {2020},
pages = {3546-3556},
volume = {108},
url = {https://mlanthology.org/aistats/2020/meehan2020aistats-three/}
}