A Probabilistic Approach to Feature Selection - A Filter Solution

Abstract

Feature selection can be defined as a problem of finding a minimum set of M relevant attributes that describes the dataset as well as the original N attributes do, where M N . After examining the problems with both the exhaustive and the heuristic approach to feature selection, this paper proposes a probabilistic approach. The theoretic analysis and the experimental study show that the proposed approach is simple to implement and guaranteed to find the optimal if resources permit. It is also fast in obtaining results and effective in selecting features that improve the performance of a learning algorithm. An on-site application involving huge datasets has been conducted independently. It proves the effectiveness and scalability of the proposed algorithm. Discussed also are various aspects and applications of this feature selection algorithm. 1 Introduction The problem of feature selection can be defined as finding M relevant attributes among the N original attrib...