Sufficiency, Separability and Temporal Probabilistic Models

Abstract

Suppose we are given the conditional probability of one variable given some other variables. Normally the full joint distribution over the conditioning variables is required to determine the probability of the conditioned variable. Under what circumstances are the marginal distributions over the conditioning variables sufficient to determine the probability of the conditioned variable? Sufficiency in this sense is equivalent to additive separability of the conditional probability distribution. Such separability structure is natural and can be exploited for efficient inference. Separability has a natural generalization to conditional separability. Separability provides a precise notion of hierarchical decomposition in temporal probabilistic models. Given a system that is decomposed into separable subsystems, exact marginal probabilities over subsystems at future points in time can be computed by propagating maxginal subsystem probabilities, rather than complete system joint probabilities. Thus, separability can make exact prediction tractable. However, observations can break separability, so exact monitoring of dynamic systems remains hard.