Differentiable Local Intrinsic Dimension Estimation with Diffusion Models
Abstract
High-dimensional data commonly lies on low-dimensional submanifolds, and estimating the local intrinsic dimension (LID) of a datum is a longstanding problem. LID can be understood as the number of local factors of variation: the more factors of variation a datum has, the more complex it tends to be. Estimating this quantity has proven useful in contexts ranging from generalization in neural networks to detection of out-of-distribution data, adversarial examples, and AI-generated text. While many estimation techniques exist, they are all either inaccurate or do not scale. In this work, we show that the Fokker-Planck equation associated with a diffusion model can provide the first LID estimator which scales to high dimensional data while outperforming existing baselines on LID estimation benchmarks.
Cite
Text
Kamkari et al. "Differentiable Local Intrinsic Dimension Estimation with Diffusion Models." ICML 2024 Workshops: Differentiable_Almost_Everything, 2024.Markdown
[Kamkari et al. "Differentiable Local Intrinsic Dimension Estimation with Diffusion Models." ICML 2024 Workshops: Differentiable_Almost_Everything, 2024.](https://mlanthology.org/icmlw/2024/kamkari2024icmlw-differentiable/)BibTeX
@inproceedings{kamkari2024icmlw-differentiable,
title = {{Differentiable Local Intrinsic Dimension Estimation with Diffusion Models}},
author = {Kamkari, Hamidreza and Ross, Brendan Leigh and Hosseinzadeh, Rasa and Cresswell, Jesse C. and Loaiza-Ganem, Gabriel},
booktitle = {ICML 2024 Workshops: Differentiable_Almost_Everything},
year = {2024},
url = {https://mlanthology.org/icmlw/2024/kamkari2024icmlw-differentiable/}
}