Curse of Slicing: Why Sliced Mutual Information Is a Deceptive Measure of Statistical Dependence
Abstract
Sliced Mutual Information (SMI) is widely used as a scalable alternative to mutual information for measuring non-linear statistical dependence. Despite its advantages, such as faster convergence, robustness to high dimensionality, and nullification only under statistical independence, we demonstrate that SMI is highly susceptible to data manipulation and exhibits counterintuitive behavior. Through extensive benchmarking and theoretical analysis, we show that SMI saturates easily, fails to detect increases in statistical dependence (even under linear transformations designed to enhance the extraction of information), prioritizes redundancy over informative content, and in some cases, performs worse than simpler dependence measures like the correlation coefficient.
Cite
Text
Semenenko et al. "Curse of Slicing: Why Sliced Mutual Information Is a Deceptive Measure of Statistical Dependence." International Conference on Learning Representations, 2026.Markdown
[Semenenko et al. "Curse of Slicing: Why Sliced Mutual Information Is a Deceptive Measure of Statistical Dependence." International Conference on Learning Representations, 2026.](https://mlanthology.org/iclr/2026/semenenko2026iclr-curse/)BibTeX
@inproceedings{semenenko2026iclr-curse,
title = {{Curse of Slicing: Why Sliced Mutual Information Is a Deceptive Measure of Statistical Dependence}},
author = {Semenenko, Alexander and Butakov, Ivan and Oseledets, Ivan and Frolov, Alexey},
booktitle = {International Conference on Learning Representations},
year = {2026},
url = {https://mlanthology.org/iclr/2026/semenenko2026iclr-curse/}
}