ML Anthology

The Preimage of Rectifier Network Activities

Stefan Carlsson, Hossein Azizpour, Ali Sharif Razavian, Josephine Sullivan, Kevin Smith
ICLR 2017
/iclr/2017/carlsson2017iclr-preimage/

Abstract

We give a procedure for explicitely computing the complete preimage of activities of a layer in a rectifier network with fully connected layers, from knowledge of the weights in the network. The most general characterization of preimages is as piecewise linear manifolds in the input space with possibly multiple branches. This work therefore complements previous demonstrations of preimages obtained by heuristic optimization and regularization algorithms Mahendran & Vedaldi (2015; 2016) We are presently empirically evaluating the procedure and it’s ability to extract complete preimages as well as the general structure of preimage manifolds. ICLR 2017 CONFRENCE TRACK SUBMISSION: https://openreview.net/forum?id=HJcLcw9xg&noteId=HJcLcw9xg

PDF ICLR Semantic Scholar

Cite

Text

Carlsson et al. "The Preimage of Rectifier Network Activities." International Conference on Learning Representations, 2017.

Markdown

[Carlsson et al. "The Preimage of Rectifier Network Activities." International Conference on Learning Representations, 2017.](https://mlanthology.org/iclr/2017/carlsson2017iclr-preimage/)

BibTeX

@inproceedings{carlsson2017iclr-preimage,
  title     = {{The Preimage of Rectifier Network Activities}},
  author    = {Carlsson, Stefan and Azizpour, Hossein and Razavian, Ali Sharif and Sullivan, Josephine and Smith, Kevin},
  booktitle = {International Conference on Learning Representations},
  year      = {2017},
  url       = {https://mlanthology.org/iclr/2017/carlsson2017iclr-preimage/}
}