Unifying and Merging Well-Trained Deep Neural Networks for Inference Stage
Abstract
We propose a novel method to merge convolutional neural-nets for the inference stage. Given two well-trained networks that may have different architectures that handle different tasks, our method aligns the layers of the original networks and merges them into a unified model by sharing the representative codes of weights. The shared weights are further re-trained to fine-tune the performance of the merged model. The proposed method effectively produces a compact model that may run original tasks simultaneously on resource-limited devices. As it preserves the general architectures and leverages the co-used weights of well-trained networks, a substantial training overhead can be reduced to shorten the system development time. Experimental results demonstrate a satisfactory performance and validate the effectiveness of the method.
Cite
Text
Chou et al. "Unifying and Merging Well-Trained Deep Neural Networks for Inference Stage." International Joint Conference on Artificial Intelligence, 2018. doi:10.24963/IJCAI.2018/283Markdown
[Chou et al. "Unifying and Merging Well-Trained Deep Neural Networks for Inference Stage." International Joint Conference on Artificial Intelligence, 2018.](https://mlanthology.org/ijcai/2018/chou2018ijcai-unifying/) doi:10.24963/IJCAI.2018/283BibTeX
@inproceedings{chou2018ijcai-unifying,
title = {{Unifying and Merging Well-Trained Deep Neural Networks for Inference Stage}},
author = {Chou, Yi-Min and Chan, Yi-Ming and Lee, Jia-Hong and Chiu, Chih-Yi and Chen, Chu-Song},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2018},
pages = {2049-2056},
doi = {10.24963/IJCAI.2018/283},
url = {https://mlanthology.org/ijcai/2018/chou2018ijcai-unifying/}
}