Exploiting Phonological Constraints for Handshape Inference in ASL Video
Abstract
Handshape is a key linguistic component of signs, and thus, handshape recognition is essential to algorithms for sign language recognition and retrieval. In this work, linguistic constraints on the relationship between start and end handshapes are leveraged to improve handshape recognition accuracy. A Bayesian network formulation is proposed for learning and exploiting these constraints, while taking into consideration inter-signer variations in the production of particular handshapes. A Variational Bayes formulation is employed for supervised learning of the model parameters. A non-rigid image alignment algorithm, which yields improved robustness to variability in handshape appearance, is proposed for computing image observation likelihoods in the model. The resulting handshape inference algorithm is evaluated using a dataset of 1500 lexical signs in American Sign Language (ASL), where each lexical sign is produced by three native ASL signers.
Cite
Text
Thangali et al. "Exploiting Phonological Constraints for Handshape Inference in ASL Video." IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011. doi:10.1109/CVPR.2011.5995718Markdown
[Thangali et al. "Exploiting Phonological Constraints for Handshape Inference in ASL Video." IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2011.](https://mlanthology.org/cvpr/2011/thangali2011cvpr-exploiting/) doi:10.1109/CVPR.2011.5995718BibTeX
@inproceedings{thangali2011cvpr-exploiting,
title = {{Exploiting Phonological Constraints for Handshape Inference in ASL Video}},
author = {Thangali, Ashwin and Nash, Joan P. and Sclaroff, Stan and Neidle, Carol},
booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year = {2011},
pages = {521-528},
doi = {10.1109/CVPR.2011.5995718},
url = {https://mlanthology.org/cvpr/2011/thangali2011cvpr-exploiting/}
}