Generative Semi-Supervised Learning with a Neural Seq2seq Noisy Channel
Abstract
We use a neural noisy channel generative model to learn the relationship between two sequences, for example text and speech, from little paired data. We identify time locality as a key assumption which is restrictive enough to support semi-supervised learning but general enough to be widely applicable. Experimentally we show that our approach is capable of recovering the relationship between written and spoken language (represented as graphemes and phonemes) from only 5 minutes of paired data. Our results pave the way for more widespread adoption of generative semi-supervised learning for seq2seq tasks.
Cite
Text
Mariooryad et al. "Generative Semi-Supervised Learning with a Neural Seq2seq Noisy Channel." ICML 2023 Workshops: SPIGM, 2023.Markdown
[Mariooryad et al. "Generative Semi-Supervised Learning with a Neural Seq2seq Noisy Channel." ICML 2023 Workshops: SPIGM, 2023.](https://mlanthology.org/icmlw/2023/mariooryad2023icmlw-generative/)BibTeX
@inproceedings{mariooryad2023icmlw-generative,
title = {{Generative Semi-Supervised Learning with a Neural Seq2seq Noisy Channel}},
author = {Mariooryad, Soroosh and Shannon, Matt and Ma, Siyuan and Bagby, Tom and Kao, David Teh-Hwa and Stanton, Daisy and Battenberg, Eric and Skerry-Ryan, Rj},
booktitle = {ICML 2023 Workshops: SPIGM},
year = {2023},
url = {https://mlanthology.org/icmlw/2023/mariooryad2023icmlw-generative/}
}