Effective Version Space Reduction for Convolutional Neural Networks

Jiayu Liu, Ioannis Chiotellis, Rudolph Triebel, Daniel Cremers

ECML-PKDD 2020 pp. 85-100

doi:10.1007/978-3-030-67661-2_6 /ecmlpkdd/2020/liu2020ecmlpkdd-effective/

Abstract

In active learning, sampling bias could pose a serious inconsistency problem and hinder the algorithm from finding the optimal hypothesis. However, many methods for neural networks are hypothesis space agnostic and do not address this problem. We examine active learning with convolutional neural networks through the principled lens of version space reduction. We identify the connection between two approaches---prior mass reduction and diameter reduction---and propose a new diameter-based querying method---the minimum Gibbs-vote disagreement. By estimating version space diameter and bias, we illustrate how version space of neural networks evolves and examine the realizability assumption. With experiments on MNIST, Fashion-MNIST, SVHN and STL-10 datasets, we demonstrate that diameter reduction methods reduce the version space more effectively and perform better than prior mass reduction and other baselines, and that the Gibbs vote disagreement is on par with the best query method.

PDF ECML-PKDD Semantic Scholar

Cite

Text

Liu et al. "Effective Version Space Reduction for Convolutional Neural Networks." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020. doi:10.1007/978-3-030-67661-2_6

Markdown

[Liu et al. "Effective Version Space Reduction for Convolutional Neural Networks." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2020.](https://mlanthology.org/ecmlpkdd/2020/liu2020ecmlpkdd-effective/) doi:10.1007/978-3-030-67661-2_6

BibTeX

@inproceedings{liu2020ecmlpkdd-effective,
  title     = {{Effective Version Space Reduction for Convolutional Neural Networks}},
  author    = {Liu, Jiayu and Chiotellis, Ioannis and Triebel, Rudolph and Cremers, Daniel},
  booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
  year      = {2020},
  pages     = {85-100},
  doi       = {10.1007/978-3-030-67661-2_6},
  url       = {https://mlanthology.org/ecmlpkdd/2020/liu2020ecmlpkdd-effective/}
}