SI-VDNAS: Semi-Implicit Variational Dropout for Hierarchical One-Shot Neural Architecture Search
Abstract
Bayesian methods have improved the interpretability and stability of neural architecture search (NAS). In this paper, we propose a novel probabilistic approach, namely Semi-Implicit Variational Dropout one-shot Neural Architecture Search (SI-VDNAS), that leverages semi-implicit variational dropout to support architecture search with variable operations and edges. SI-VDNAS achieves stable training that would not be affected by the over-selection of skip-connect operation. Experimental results demonstrate that SI-VDNAS finds a convergent architecture with only 2.7 MB parameters within 0.8 GPU-days and can achieve 2.60% top-1 error rate on CIFAR-10. The convergent architecture can obtain a top-1 error rate of 16.20% and 25.6% when transferred to CIFAR-100 and ImageNet (mobile setting).
Cite
Text
Wang et al. "SI-VDNAS: Semi-Implicit Variational Dropout for Hierarchical One-Shot Neural Architecture Search." International Joint Conference on Artificial Intelligence, 2020. doi:10.24963/IJCAI.2020/289Markdown
[Wang et al. "SI-VDNAS: Semi-Implicit Variational Dropout for Hierarchical One-Shot Neural Architecture Search." International Joint Conference on Artificial Intelligence, 2020.](https://mlanthology.org/ijcai/2020/wang2020ijcai-si/) doi:10.24963/IJCAI.2020/289BibTeX
@inproceedings{wang2020ijcai-si,
title = {{SI-VDNAS: Semi-Implicit Variational Dropout for Hierarchical One-Shot Neural Architecture Search}},
author = {Wang, Yaoming and Dai, Wenrui and Li, Chenglin and Zou, Junni and Xiong, Hongkai},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2020},
pages = {2088-2095},
doi = {10.24963/IJCAI.2020/289},
url = {https://mlanthology.org/ijcai/2020/wang2020ijcai-si/}
}