PAC Learning from Positive Statistical Queries

Abstract

Learning from positive examples occurs very frequently in natural learning. The PAC learning model of Valiant takes many features of natural learning into account, but in most cases it fails to describe such kind of learning. We show that in order to make the learning from positive data possible, extra-information about the underlying distribution must be provided to the learner. We define a PAC learning model from positive and unlabeled examples. We also define a PAC learning model from positive and unlabeled statistical queries. Relations with PAC model ([ Val84 ]), statistical query model ([ Kea93 ]) and constant-partition classification noise model ([ Dec97 ]) are studied. We show that k DNF and k decision lists are learnable in both models, i.e. with far less information than it is assumed in previously used algorithms.