Datum-Wise Classification: A Sequential Approach to Sparsity
Abstract
We propose a novel classification technique whose aim is to select an appropriate representation for each datapoint, in contrast to the usual approach of selecting a representation encompassing the whole dataset. This datum-wise representation is found by using a sparsity inducing empirical risk, which is a relaxation of the standard L0 regularized risk. The classification problem is modeled as a sequential decision process that sequentially chooses, for each datapoint, which features to use before classifying. Datum-Wise Classification extends naturally to multi-class tasks, and we describe a specific case where our inference has equivalent complexity to a traditional linear classifier, while still using a variable number of features. We compare our classifier to classical L1 regularized linear models (L1-SVM and LARS) on a set of common binary and multi-class datasets and show that for an equal average number of features used we can get improved performance using our method.
Cite
Text
Dulac-Arnold et al. "Datum-Wise Classification: A Sequential Approach to Sparsity." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2011. doi:10.1007/978-3-642-23780-5_34Markdown
[Dulac-Arnold et al. "Datum-Wise Classification: A Sequential Approach to Sparsity." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2011.](https://mlanthology.org/ecmlpkdd/2011/dulacarnold2011ecmlpkdd-datumwise/) doi:10.1007/978-3-642-23780-5_34BibTeX
@inproceedings{dulacarnold2011ecmlpkdd-datumwise,
title = {{Datum-Wise Classification: A Sequential Approach to Sparsity}},
author = {Dulac-Arnold, Gabriel and Denoyer, Ludovic and Preux, Philippe and Gallinari, Patrick},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2011},
pages = {375-390},
doi = {10.1007/978-3-642-23780-5_34},
url = {https://mlanthology.org/ecmlpkdd/2011/dulacarnold2011ecmlpkdd-datumwise/}
}