Associative Variational Auto-Encoder with Distributed Latent Spaces and Associators
Abstract
In this paper, we propose a novel structure for a multi-modal data association referred to as Associative Variational Auto-Encoder (AVAE). In contrast to the existing models using a shared latent space among modalities, our structure adopts distributed latent spaces for multi-modalities which are connected through cross-modal associators. The proposed structure successfully associates even heterogeneous modality data and easily incorporates the additional modality to the entire network via the associator. Furthermore, in our structure, only a small amount of supervised (paired) data is enough to train associators after training auto-encoders in an unsupervised manner. Through experiments, the effectiveness of the proposed structure is validated on various datasets including visual and auditory data.
Cite
Text
Jo et al. "Associative Variational Auto-Encoder with Distributed Latent Spaces and Associators." AAAI Conference on Artificial Intelligence, 2020. doi:10.1609/AAAI.V34I07.6778Markdown
[Jo et al. "Associative Variational Auto-Encoder with Distributed Latent Spaces and Associators." AAAI Conference on Artificial Intelligence, 2020.](https://mlanthology.org/aaai/2020/jo2020aaai-associative/) doi:10.1609/AAAI.V34I07.6778BibTeX
@inproceedings{jo2020aaai-associative,
title = {{Associative Variational Auto-Encoder with Distributed Latent Spaces and Associators}},
author = {Jo, Dae Ung and Lee, Byeongju and Choi, Jongwon and Yoo, Haanju and Choi, Jin Young},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2020},
pages = {11197-11204},
doi = {10.1609/AAAI.V34I07.6778},
url = {https://mlanthology.org/aaai/2020/jo2020aaai-associative/}
}