Real-Time Sign Language Detection Using Human Pose Estimation
Abstract
We propose a lightweight real-time sign language detection model, as we identify the need for such a case in videoconferencing. We extract optical flow features based on human pose estimation and, using a linear classifier, show these features are meaningful with an accuracy of 80%, evaluated on the DGS Corpus. Using a recurrent model directly on the input, we see improvements of up to 91% accuracy, while still working under 4ms. We describe a demo application to sign language detection in the browser in order to demonstrate its usage possibility in videoconferencing applications.
Cite
Text
Moryossef et al. "Real-Time Sign Language Detection Using Human Pose Estimation." European Conference on Computer Vision Workshops, 2020. doi:10.1007/978-3-030-66096-3_17Markdown
[Moryossef et al. "Real-Time Sign Language Detection Using Human Pose Estimation." European Conference on Computer Vision Workshops, 2020.](https://mlanthology.org/eccvw/2020/moryossef2020eccvw-realtime/) doi:10.1007/978-3-030-66096-3_17BibTeX
@inproceedings{moryossef2020eccvw-realtime,
title = {{Real-Time Sign Language Detection Using Human Pose Estimation}},
author = {Moryossef, Amit and Tsochantaridis, Ioannis and Aharoni, Roee and Ebling, Sarah and Narayanan, Srini},
booktitle = {European Conference on Computer Vision Workshops},
year = {2020},
pages = {237-248},
doi = {10.1007/978-3-030-66096-3_17},
url = {https://mlanthology.org/eccvw/2020/moryossef2020eccvw-realtime/}
}