Dropout Regularization in Extended Generalized Linear Models Based on Double Exponential Families

Schwienhorst, Benedikt Lütke; Kock, Lucas; Klein, Nadja; Nott, David J.

doi:10.1007/978-3-031-70365-2_19

Dropout Regularization in Extended Generalized Linear Models Based on Double Exponential Families

Benedikt Lütke Schwienhorst, Lucas Kock, Nadja Klein, David J. Nott

ECML-PKDD 2024 pp. 320-336

doi:10.1007/978-3-031-70365-2_19 /ecmlpkdd/2024/schwienhorst2024ecmlpkdd-dropout/

Abstract

Even though dropout is a popular regularization technique, its theoretical properties are not fully understood. In this paper we study dropout regularization in extended generalized linear models based on double exponential families, for which the dispersion parameter can vary with the features. A theoretical analysis shows that dropout regularization prefers rare but important features in both the mean and dispersion, generalizing an earlier result for conventional generalized linear models. To illustrate, we apply dropout to adaptive smoothing with B-splines, where both the mean and dispersion parameters are modeled flexibly. The important B-spline basis functions can be thought of as rare features, and we confirm in experiments that dropout is an effective form of regularization for mean and dispersion parameters that improves on a penalized maximum likelihood approach with an explicit smoothness penalty. An application to traffic detection data from Berlin further illustrates the benefits of our method.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Schwienhorst et al. "Dropout Regularization in Extended Generalized Linear Models Based on Double Exponential Families." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2024. doi:10.1007/978-3-031-70365-2_19

Markdown

[Schwienhorst et al. "Dropout Regularization in Extended Generalized Linear Models Based on Double Exponential Families." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2024.](https://mlanthology.org/ecmlpkdd/2024/schwienhorst2024ecmlpkdd-dropout/) doi:10.1007/978-3-031-70365-2_19

BibTeX

@inproceedings{schwienhorst2024ecmlpkdd-dropout,
  title     = {{Dropout Regularization in Extended Generalized Linear Models Based on Double Exponential Families}},
  author    = {Schwienhorst, Benedikt Lütke and Kock, Lucas and Klein, Nadja and Nott, David J.},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2024},
  pages     = {320-336},
  doi       = {10.1007/978-3-031-70365-2_19},
  url       = {https://mlanthology.org/ecmlpkdd/2024/schwienhorst2024ecmlpkdd-dropout/}
}