Defining and Quantifying the Emergence of Sparse Concepts in DNNs

Jie Ren, Mingjie Li, Qirui Chen, Huiqi Deng, Quanshi Zhang

CVPR 2023 pp. 20280-20289

doi:10.1109/CVPR52729.2023.01942 /cvpr/2023/ren2023cvpr-defining/

Abstract

This paper aims to illustrate the concept-emerging phenomenon in a trained DNN. Specifically, we find that the inference score of a DNN can be disentangled into the effects of a few interactive concepts. These concepts can be understood as inference patterns in a sparse, symbolic graphical model, which explains the DNN. The faithfulness of using such a graphical model to explain the DNN is theoretically guaranteed, because we prove that the graphical model can well mimic the DNN's outputs on an exponential number of different masked samples. Besides, such a graphical model can be further simplified and re-written as an And-Or graph (AOG), without losing much explanation accuracy. The code is released at https://github.com/sjtu-xai-lab/aog.

PDF CVPR Semantic Scholar

Cite

Text

Ren et al. "Defining and Quantifying the Emergence of Sparse Concepts in DNNs." Conference on Computer Vision and Pattern Recognition, 2023. doi:10.1109/CVPR52729.2023.01942

Markdown

[Ren et al. "Defining and Quantifying the Emergence of Sparse Concepts in DNNs." Conference on Computer Vision and Pattern Recognition, 2023.](https://mlanthology.org/cvpr/2023/ren2023cvpr-defining/) doi:10.1109/CVPR52729.2023.01942

BibTeX

@inproceedings{ren2023cvpr-defining,
  title     = {{Defining and Quantifying the Emergence of Sparse Concepts in DNNs}},
  author    = {Ren, Jie and Li, Mingjie and Chen, Qirui and Deng, Huiqi and Zhang, Quanshi},
  booktitle = {Conference on Computer Vision and Pattern Recognition},
  year      = {2023},
  pages     = {20280-20289},
  doi       = {10.1109/CVPR52729.2023.01942},
  url       = {https://mlanthology.org/cvpr/2023/ren2023cvpr-defining/}
}