Enhancing Uncertainty Estimation and Interpretability with Bayesian Non-Negative Decision Layer

Abstract

Although deep neural networks have demonstrated significant success due to their powerful expressiveness, most models struggle to meet practical requirements for uncertainty estimation. Concurrently, the entangled nature of deep neural net- works leads to a multifaceted problem, where various localized explanation tech- niques reveal that multiple unrelated features influence the decisions, thereby un- dermining interpretability. To address these challenges, we develop a Bayesian Nonnegative Decision Layer (BNDL), which reformulates deep neural networks as a conditional Bayesian non-negative factor analysis. By leveraging stochastic latent variables, the BNDL can model complex dependencies and provide robust uncertainty estimation. Moreover, the sparsity and non-negativity of the latent variables encourage the model to learn disentangled representations and decision layers, thereby improving interpretability. We also offer theoretical guarantees that BNDL can achieve effective disentangled learning. In addition, we developed a corresponding variational inference method utilizing a Weibull variational in- ference network to approximate the posterior distribution of the latent variables. Our experimental results demonstrate that with enhanced disentanglement capa- bilities, BNDL not only improves the model’s accuracy but also provides reliable uncertainty estimation and improved interpretability.