Multi-Model Evaluation with Labeled & Unlabeled Data
Abstract
It remains difficult to select a machine learning model from a set of candidates in the absence of a large, labeled dataset. To address this challenge, we propose a framework to compare multiple models that leverages three aspects of modern machine learning settings: multiple machine learning classifiers, continuous predictions on all examples, and abundant unlabeled data. The key idea is to estimate the joint distribution of classifier predictions using a mixture model, where each component corresponds to a different class. We present preliminary experiments on a large health dataset and conclude with future directions.
Cite
Text
Shanmugam et al. "Multi-Model Evaluation with Labeled & Unlabeled Data." ICLR 2024 Workshops: PML4LRS, 2024.Markdown
[Shanmugam et al. "Multi-Model Evaluation with Labeled & Unlabeled Data." ICLR 2024 Workshops: PML4LRS, 2024.](https://mlanthology.org/iclrw/2024/shanmugam2024iclrw-multimodel/)BibTeX
@inproceedings{shanmugam2024iclrw-multimodel,
title = {{Multi-Model Evaluation with Labeled & Unlabeled Data}},
author = {Shanmugam, Divya M and Sadhuka, Shuvom and Raghavan, Manish and Guttag, John and Berger, Bonnie and Pierson, Emma},
booktitle = {ICLR 2024 Workshops: PML4LRS},
year = {2024},
url = {https://mlanthology.org/iclrw/2024/shanmugam2024iclrw-multimodel/}
}