Feature Selection with Selective Sampling

Abstract

Feature selection, as a preprocessing step to machine learning, has been shown very effective in reducing dimensionality, removing irrelevant data, increasing learning accuracy, and improving comprehensibility. In this paper, we consider the problem of active feature selection in a lter model setting. We describe a formalism of active feature selection called selective sampling, demonstrate it by applying it to a widely used feature selection algorithm Relief, and show how it realizes active feature selection and reduces the required number of training data for Relief to achieve time savings without performance deterioration. We design objective evaluation measures, conduct extensive experiments using bench-mark data sets, and observe consistent and signi cant improvement.