ML Anthology

Talking with Signs: A Simple Method to Detect Nouns and Numbers in a Non-Annotated Signs Language Corpus

Eric Raphael Huiza Pereyra, Cesar Augusto Olivares Poggi
CVPRW 2021 pp. 1212-1220
doi:10.1109/CVPRW53098.2021.00132 /cvprw/2021/pereyra2021cvprw-talking/

Abstract

People with deafness or hearing disabilities who aim to use computer based systems rely on state-of-art video classification and human action recognition techniques that combine traditional movement pattern recognition and deep learning techniques. In this work we present a pipeline for semi-automatic video annotation applied to a non-annotated Peruvian Signs Language (PSL) corpus along with a novel method for a progressive detection of PSL elements (nSDm). We produced a set of video annotations indicating signs appearances for a small set of nouns and numbers along with a labeled PSL dataset (PSL dataset). A model obtained after ensemble a 2D CNN trained with movement patterns extracted from the PSL dataset using Lucas Kanade Opticalflow, and a RNN with LSTM cells trained with raw RGB frames extracted from the PSL dataset reporting state-of-art results over the PSL dataset on signs classification tasks in terms of AUC, Precision and Recall.

PDF CVPRW Semantic Scholar

Cite

Text

Pereyra and Poggi. "Talking with Signs: A Simple Method to Detect Nouns and Numbers in a Non-Annotated Signs Language Corpus." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2021. doi:10.1109/CVPRW53098.2021.00132

Markdown

[Pereyra and Poggi. "Talking with Signs: A Simple Method to Detect Nouns and Numbers in a Non-Annotated Signs Language Corpus." IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2021.](https://mlanthology.org/cvprw/2021/pereyra2021cvprw-talking/) doi:10.1109/CVPRW53098.2021.00132

BibTeX

@inproceedings{pereyra2021cvprw-talking,
  title     = {{Talking with Signs: A Simple Method to Detect Nouns and Numbers in a Non-Annotated Signs Language Corpus}},
  author    = {Pereyra, Eric Raphael Huiza and Poggi, Cesar Augusto Olivares},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year      = {2021},
  pages     = {1212-1220},
  doi       = {10.1109/CVPRW53098.2021.00132},
  url       = {https://mlanthology.org/cvprw/2021/pereyra2021cvprw-talking/}
}