Heavy-Tailed Kernels Reveal a Finer Cluster Structure in T-SNE Visualisations

Dmitry Kobak, George C. Linderman, Stefan Steinerberger, Yuval Kluger, Philipp Berens

ECML-PKDD 2019 pp. 124-139

doi:10.1007/978-3-030-46150-8_8 /ecmlpkdd/2019/kobak2019ecmlpkdd-heavytailed/

Abstract

T-distributed stochastic neighbour embedding (t-SNE) is a widely used data visualisation technique. It differs from its predecessor SNE by the low-dimensional similarity kernel: the Gaussian kernel was replaced by the heavy-tailed Cauchy kernel, solving the 'crowding problem' of SNE. Here, we develop an efficient implementation of t-SNE for a t-distribution kernel with an arbitrary degree of freedom ν, with ν → ∞ corresponding to SNE and ν = 1 corresponding to the standard t-SNE. Using theoretical analysis and toy examples, we show that ν < 1 can further reduce the crowding problem and reveal finer cluster structure that is invisible in standard t-SNE. We further demonstrate the striking effect of heavier-tailed kernels on large real-life data sets such as MNIST, single-cell RNA-sequencing data, and the HathiTrust library. We use domain knowledge to confirm that the revealed clusters are meaningful. Overall, we argue that modifying the tail heaviness of the t-SNE kernel can yield additional insight into the cluster structure of the data.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Kobak et al. "Heavy-Tailed Kernels Reveal a Finer Cluster Structure in T-SNE Visualisations." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019. doi:10.1007/978-3-030-46150-8_8

Markdown

[Kobak et al. "Heavy-Tailed Kernels Reveal a Finer Cluster Structure in T-SNE Visualisations." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019.](https://mlanthology.org/ecmlpkdd/2019/kobak2019ecmlpkdd-heavytailed/) doi:10.1007/978-3-030-46150-8_8

BibTeX

@inproceedings{kobak2019ecmlpkdd-heavytailed,
  title     = {{Heavy-Tailed Kernels Reveal a Finer Cluster Structure in T-SNE Visualisations}},
  author    = {Kobak, Dmitry and Linderman, George C. and Steinerberger, Stefan and Kluger, Yuval and Berens, Philipp},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2019},
  pages     = {124-139},
  doi       = {10.1007/978-3-030-46150-8_8},
  url       = {https://mlanthology.org/ecmlpkdd/2019/kobak2019ecmlpkdd-heavytailed/}
}