Model Selection Criteria for Learning Belief Nets: An Empirical Comparison

Abstract

Learning the dependency structure of a (Bayesian) belief net involves a trade-o between simplicity and goodness of t to the training data. We describe the results of an empirical comparison of three standard model selection criteria | viz., a Minimum Description Length criterion (MDL), Akaike's Information Criterion (AIC) and a Cross-Validation criterion (XV) | applied to this problem. Our results suggest that AIC and XV are both good criteria for avoiding overtting, but MDL does not work well in this context. This report focuses on the challenge of learning the (Bayesian) belief net BN [Pea88] that has minimum KL-divergence [KL51] from the true distribution, D over a set of discrete variables X | i.e., the network that minimizes 1 info( BN ; D ) = X x PD ( X = x ) log PBN ( X = x ) from a xed training sample s drawn iid from D. As it is easy to nd the optimal parameter values (i.e., \\CPtable entries") for a given structure [CH92, Hec95], we focus further on sel...