Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition
Abstract
This work presents a framework to recognise signer independent mouthings in continuous sign language, with no manual annotations needed. Mouthings represent lip-movements that correspond to pronunciations of words or parts of them during signing. Research on sign language recognition has focused extensively on the hands as features. But sign language is multi-modal and a full understanding particularly with respect to its lexical variety, language idioms and grammatical structures is not possible without further exploring the remaining information channels. To our knowledge no previous work has explored dedicated viseme recognition in the context of sign language recognition. The approach is trained on over 180.000 unlabelled frames and reaches 47.1% precision on the frame level. Generalisation across individuals and the influence of context-dependent visemes are analysed.
Cite
Text
Koller et al. "Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition." European Conference on Computer Vision, 2014. doi:10.1007/978-3-319-10590-1_19Markdown
[Koller et al. "Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition." European Conference on Computer Vision, 2014.](https://mlanthology.org/eccv/2014/koller2014eccv-read/) doi:10.1007/978-3-319-10590-1_19BibTeX
@inproceedings{koller2014eccv-read,
title = {{Read My Lips: Continuous Signer Independent Weakly Supervised Viseme Recognition}},
author = {Koller, Oscar and Ney, Hermann and Bowden, Richard},
booktitle = {European Conference on Computer Vision},
year = {2014},
pages = {281-296},
doi = {10.1007/978-3-319-10590-1_19},
url = {https://mlanthology.org/eccv/2014/koller2014eccv-read/}
}