Sobolev Training with Approximated Derivatives for Black-Box Function Regression with Neural Networks
Abstract
With Sobolev Training, neural networks are trained to fit target output values as well as target derivatives with respect to the inputs. This leads to better generalization and fewer required training examples for certain problems. In this paper, we present a training pipeline that enables Sobolev Training for regression problems where target derivatives are not directly available. Thus, we propose to use a least-squares estimate of the target derivatives based on function values of neighboring training samples. We show for a variety of black-box function regression tasks that our training pipeline achieves smaller test errors compared to the traditional training method. Since our method has no additional requirements on the data collection process, it has great potential to improve the results for various regression tasks.
Cite
Text
Kissel and Diepold. "Sobolev Training with Approximated Derivatives for Black-Box Function Regression with Neural Networks." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019. doi:10.1007/978-3-030-46147-8_24Markdown
[Kissel and Diepold. "Sobolev Training with Approximated Derivatives for Black-Box Function Regression with Neural Networks." European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, 2019.](https://mlanthology.org/ecmlpkdd/2019/kissel2019ecmlpkdd-sobolev/) doi:10.1007/978-3-030-46147-8_24BibTeX
@inproceedings{kissel2019ecmlpkdd-sobolev,
title = {{Sobolev Training with Approximated Derivatives for Black-Box Function Regression with Neural Networks}},
author = {Kissel, Matthias and Diepold, Klaus},
booktitle = {European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases},
year = {2019},
pages = {399-414},
doi = {10.1007/978-3-030-46147-8_24},
url = {https://mlanthology.org/ecmlpkdd/2019/kissel2019ecmlpkdd-sobolev/}
}