The Differentiable Curry
Abstract
We revisit the automatic differentiation (AD) of programs that contain higher-order functions, in a statically typed setting. Our presentation builds on a recent formulation of AD based on categorical combinators, and shows how that formulation can be extended to higher-order functions via two different notions of differentiable currying and evaluation. We present these alternative implementations, and justify their correctness by means of showing that AD yields equivalent back-propagators for forward-equivalent programs, even in the presence of higher-order features. Higher-order functions complicate the definition of equivalence for back-propagators compared to simpler forms of program equivalence.
Cite
Text
Vytiniotis et al. "The Differentiable Curry." NeurIPS 2019 Workshops: Program_Transformations, 2019.Markdown
[Vytiniotis et al. "The Differentiable Curry." NeurIPS 2019 Workshops: Program_Transformations, 2019.](https://mlanthology.org/neuripsw/2019/vytiniotis2019neuripsw-differentiable/)BibTeX
@inproceedings{vytiniotis2019neuripsw-differentiable,
title = {{The Differentiable Curry}},
author = {Vytiniotis, Dimitrios and Belov, Dan and Wei, Richard and Plotkin, Gordon and Abadi, Martin},
booktitle = {NeurIPS 2019 Workshops: Program_Transformations},
year = {2019},
url = {https://mlanthology.org/neuripsw/2019/vytiniotis2019neuripsw-differentiable/}
}