ML Anthology

Landmark-Based Ensemble Learning with Random Fourier Features and Gradient Boosting

Léo Gautheron, Pascal Germain, Amaury Habrard, Guillaume Metzler, Emilie Morvant, Marc Sebban, Valentina Zantedeschi
ECML-PKDD 2020 pp. 141-157
doi:10.1007/978-3-030-67664-3_9 /ecmlpkdd/2020/gautheron2020ecmlpkdd-landmarkbased/

Abstract

This paper jointly leverages two state-of-the-art learning stra-tegies—gradient boosting (GB) and kernel Random Fourier Features (RFF)—to address the problem of kernel learning. Our study builds on a recent result showing that one can learn a distribution over the RFF to produce a new kernel suited for the task at hand. For learning this distribution, we exploit a GB scheme expressed as ensembles of RFF weak learners, each of them being a kernel function designed to fit the residual. Unlike Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it from the training data as a weighted sum of RFF. This strategy allows one to build a classifier based on a small ensemble of learned kernel “landmarks” better suited for the underlying application. We conduct a thorough experimental analysis to highlight the advantages of our method compared to both boosting-based and kernel-learning state-of-the-art methods.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Gautheron et al. "Landmark-Based Ensemble Learning with Random Fourier Features and Gradient Boosting." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020. doi:10.1007/978-3-030-67664-3_9

Markdown

[Gautheron et al. "Landmark-Based Ensemble Learning with Random Fourier Features and Gradient Boosting." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020.](https://mlanthology.org/ecmlpkdd/2020/gautheron2020ecmlpkdd-landmarkbased/) doi:10.1007/978-3-030-67664-3_9

BibTeX

@inproceedings{gautheron2020ecmlpkdd-landmarkbased,
  title     = {{Landmark-Based Ensemble Learning with Random Fourier Features and Gradient Boosting}},
  author    = {Gautheron, Léo and Germain, Pascal and Habrard, Amaury and Metzler, Guillaume and Morvant, Emilie and Sebban, Marc and Zantedeschi, Valentina},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2020},
  pages     = {141-157},
  doi       = {10.1007/978-3-030-67664-3_9},
  url       = {https://mlanthology.org/ecmlpkdd/2020/gautheron2020ecmlpkdd-landmarkbased/}
}