The Problem of Induction and Machine Learning

Abstract

Are we justified in inferring a general rule from observations that frequently confirm it? This is the usual statement of the problem of induction. The present paper argues that this question is relevant for the understanding of Machine Learning, but insufficient. Research in Machine Learning has prompted another, more fundamental question: the number of possible rules grows exponentially with the size of the examples, and many of them are somehow confirmed by the data - how are we to choose effectively some rules that have good chances of being predictive? We analyze if and how this problem is approached in standard accounts of induction and show the difficulties that are present. Finally, we suggest that the Explanation-based Learning approach and related methods of knowledge intensive induction could be a partial solution to some of these problems, and help understanding the question of valid induction from a new perspective.