Differentiation of High-Level Language Semantics

Innes, Michael

Differentiation of High-Level Language Semantics

NeurIPSW 2019

/neuripsw/2019/innes2019neuripsw-differentiation/

Abstract

Though analytic differentiation (AD) is a program transformation, AD tools have typically supported only very limited program representations, consisting of primitive mathematical operations and basic structured control flow. Zygote, an AD for the Julia language, instead operates on Julia code. This presents an interesting challenge for the AD implementor: the program representation now contains not just mathematical operations, but arbitrary control flow, user-defined functions, recursion, data structures, mutation, metaprogramming, foreign function calls, specialised hardware, and even concurrency and parallelism primitives. This paper explains how Zygote handles these high-level features safely and efficiently, making an unusually large set of Julia programs differentiable.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Innes. "Differentiation of High-Level Language Semantics." NeurIPS 2019 Workshops: Program_Transformations, 2019.

Markdown

[Innes. "Differentiation of High-Level Language Semantics." NeurIPS 2019 Workshops: Program_Transformations, 2019.](https://mlanthology.org/neuripsw/2019/innes2019neuripsw-differentiation/)

BibTeX

@inproceedings{innes2019neuripsw-differentiation,
  title     = {{Differentiation of High-Level Language Semantics}},
  author    = {Innes, Michael},
  booktitle = {NeurIPS 2019 Workshops: Program_Transformations},
  year      = {2019},
  url       = {https://mlanthology.org/neuripsw/2019/innes2019neuripsw-differentiation/}
}