Analysis of Deep Neural Networks with Extended Data Jacobian Matrix
Abstract
Deep neural networks have achieved great successes on various machine learning tasks, however, there are many open fundamental questions to be answered. In this paper, we tackle the problem of quantifying the quality of learned wights of different networks with possibly different architectures, going beyond considering the final classification error as the only metric. We introduce \emphExtended Data Jacobian Matrix to help analyze properties of networks of various structures, finding that, the spectrum of the extended data jacobian matrix is a strong discriminating factor for networks of different structures and performance. Based on such observation, we propose a novel regularization method, which manages to improve the network performance comparably to dropout, which in turn verifies the observation.
Cite
Text
Wang et al. "Analysis of Deep Neural Networks with Extended Data Jacobian Matrix." International Conference on Machine Learning, 2016.Markdown
[Wang et al. "Analysis of Deep Neural Networks with Extended Data Jacobian Matrix." International Conference on Machine Learning, 2016.](https://mlanthology.org/icml/2016/wang2016icml-analysis/)BibTeX
@inproceedings{wang2016icml-analysis,
title = {{Analysis of Deep Neural Networks with Extended Data Jacobian Matrix}},
author = {Wang, Shengjie and Mohamed, Abdel-rahman and Caruana, Rich and Bilmes, Jeff and Plilipose, Matthai and Richardson, Matthew and Geras, Krzysztof and Urban, Gregor and Aslan, Ozlem},
booktitle = {International Conference on Machine Learning},
year = {2016},
pages = {718-726},
volume = {48},
url = {https://mlanthology.org/icml/2016/wang2016icml-analysis/}
}