Demystifying Information-Theoretic Clustering
Abstract
We propose a novel method for clustering data which is grounded in information-theoretic principles and requires no parametric assumptions. Previous attempts to use information theory to define clusters in an assumption-free way are based on maximizing mutual information between data and cluster labels. We demonstrate that this intuition suffers from a fundamental conceptual flaw that causes clustering performance to deteriorate as the amount of data increases. Instead, we return to the axiomatic foundations of information theory to define a meaningful clustering measure based on the notion of consistency under coarse-graining for finite data.
Cite
Text
Ver Steeg et al. "Demystifying Information-Theoretic Clustering." International Conference on Machine Learning, 2014.Markdown
[Ver Steeg et al. "Demystifying Information-Theoretic Clustering." International Conference on Machine Learning, 2014.](https://mlanthology.org/icml/2014/versteeg2014icml-demystifying/)BibTeX
@inproceedings{versteeg2014icml-demystifying,
title = {{Demystifying Information-Theoretic Clustering}},
author = {Ver Steeg, Greg and Galstyan, Aram and Sha, Fei and DeDeo, Simon},
booktitle = {International Conference on Machine Learning},
year = {2014},
pages = {19-27},
volume = {32},
url = {https://mlanthology.org/icml/2014/versteeg2014icml-demystifying/}
}