Sign-to-Speech Model for Sign Language Understanding: A Case Study of Nigerian Sign Language
Abstract
Through this paper, we seek to reduce the communication barrier between the hearing-impaired community and the larger society who are usually not familiar with sign language in the sub-Saharan region of Africa with the largest occurrences of hearing disability cases, while using Nigeria as a case study. The dataset is a pioneer dataset for the Nigerian Sign Language and was created in collaboration with relevant stakeholders. We pre-processed the data in readiness for two different object detection models and a classification model and employed diverse evaluation metrics to gauge model performance on sign-language to text conversion tasks. Finally, we convert the predicted sign texts to speech and deploy the best performing model in a lightweight application that works in real-time and achieves impressive results converting sign words/phrases to text and subsequently, into speech.
Cite
Text
Kolawole et al. "Sign-to-Speech Model for Sign Language Understanding: A Case Study of Nigerian Sign Language." International Joint Conference on Artificial Intelligence, 2022. doi:10.24963/IJCAI.2022/855Markdown
[Kolawole et al. "Sign-to-Speech Model for Sign Language Understanding: A Case Study of Nigerian Sign Language." International Joint Conference on Artificial Intelligence, 2022.](https://mlanthology.org/ijcai/2022/kolawole2022ijcai-sign/) doi:10.24963/IJCAI.2022/855BibTeX
@inproceedings{kolawole2022ijcai-sign,
title = {{Sign-to-Speech Model for Sign Language Understanding: A Case Study of Nigerian Sign Language}},
author = {Kolawole, Steven and Osakuade, Opeyemi and Saxena, Nayan and Olorisade, Babatunde Kazeem},
booktitle = {International Joint Conference on Artificial Intelligence},
year = {2022},
pages = {5924-5927},
doi = {10.24963/IJCAI.2022/855},
url = {https://mlanthology.org/ijcai/2022/kolawole2022ijcai-sign/}
}