Why Discretization Works for Naive Bayesian Classifiers

Abstract

This paper explains why well-known discretization methods, such as entropy-based and ten-bin, work well for naive Bayesian classifiers with continuous variables, regardless of their complexities. These methods usually assume that discretized variables have Dirichlet priors. Since perfect aggregation holds for Dirichlets, we can show that, generally, a wide variety of discretization methods can perform well with insignificant difference. We identify situations where discretization may cause performance degradation and show that they are unlikely to happen for well-known methods. We empirically test our explanation with synthesized and real data sets and obtain confirming results. Our analysis leads to a lazy discretization method that can simplify the training for naive Bayes. This new method can perform as well as well-known methods in our experiment. 1. Introduction Learning a naive Bayesian classifier (a.k.a. naive Bayes) (Langley et al., 1992) from data is an ...