Automatic Differentiation: Inverse Accumulation Mode

Abstract

We show that, under certain circumstances, it is possible to automatically compute Jacobian-inverse-vector and Jacobian-inverse-transpose-vector products about as efficiently as Jacobian-vector and Jacobian-transpose-vector products. The key insight is to notice that the Jacobian corresponding to the use of one primitive arithmetic operator is of a form whose sparsity is invariant to inversion. This technique has the potential to allow the efficient direct calculation of Newton steps.

PDF NeurIPSW OpenReview Semantic Scholar

Cite

Text

Siskind. "Automatic Differentiation: Inverse Accumulation Mode." NeurIPS 2019 Workshops: Program_Transformations, 2019.

Markdown

[Siskind. "Automatic Differentiation: Inverse Accumulation Mode." NeurIPS 2019 Workshops: Program_Transformations, 2019.](https://mlanthology.org/neuripsw/2019/siskind2019neuripsw-automatic/)

BibTeX

@inproceedings{siskind2019neuripsw-automatic,
  title     = {{Automatic Differentiation: Inverse Accumulation Mode}},
  author    = {Siskind, Jeffrey Mark},
  booktitle = {NeurIPS 2019 Workshops: Program_Transformations},
  year      = {2019},
  url       = {https://mlanthology.org/neuripsw/2019/siskind2019neuripsw-automatic/}
}