Learning Causally Linked Markov Random Fields

Abstract

1 Introduction Generative models are widely used within machinelearning. However, in many applications the graphical models involve exclusively causal, or exclusivelyundirected edges. In this paper we consider models that contain both types of edge, and suggest approx-imate learning methods for such models. The main contribution of this paper is the proposal of combiningvariational inference with the contrastive divergence algorithm to facilitate learning in systems involvingcausally linked Markov Random Fields (MRF's). We support our proposal with examples of learning in sev-eral domains. 2 Learning Causal Models One way to make generative models with stochastichidden variables is to use a directed acyclic graph as shown in Figure 1 (a). The difficulty in learning such"causal " models is that the posterior distribution over the hidden variables is intractable (except in certainspecial cases such as factor analysis, mixture models, square ICA or graphs that are very sparsely con-nected). Despite the intractability of the posterior, it is possible to optimize a bound on the log proba-bility of the data by using a simple factorial distribution, Q(h|x), as an approximation to the true pos-terior,