Towards Principled Feature Selection: Relevancy, Filters and Wrappers
Abstract
In an influential paper Kohavi and John [7] presented a number of disadvantages of the filter approach to the feature selection problem, steering research towards algorithms adopting the wrapper approach. We show here that neither approach is inherently better and that any practical feature selection algorithm needs to at least consider the learner used for classification and the metric used for evaluating the learner’s performance. In the process we formally define the feature selection problem, re-examine the relationship between relevancy and filter algorithms, and establish a connection between Kohavi and John’s definition of relevancy to the Markov Blanket of a target variable in a Bayesian Network faithful to some data distribution. The theoretical results lead to principled ways of designing optimal filter algorithms of which we present one example.
Cite
Text
Tsamardinos and Aliferis. "Towards Principled Feature Selection: Relevancy, Filters and Wrappers." Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, 2003.Markdown
[Tsamardinos and Aliferis. "Towards Principled Feature Selection: Relevancy, Filters and Wrappers." Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics, 2003.](https://mlanthology.org/aistats/2003/tsamardinos2003aistats-principled/)BibTeX
@inproceedings{tsamardinos2003aistats-principled,
title = {{Towards Principled Feature Selection: Relevancy, Filters and Wrappers}},
author = {Tsamardinos, Ioannis and Aliferis, Constantin F.},
booktitle = {Proceedings of the Ninth International Workshop on Artificial Intelligence and Statistics},
year = {2003},
pages = {300-307},
volume = {R4},
url = {https://mlanthology.org/aistats/2003/tsamardinos2003aistats-principled/}
}