Abstract—Sign Language Recognition (SLR) has become an
appealing topic in modern societies because such technology
can ideally be used to bridge the gap between deaf and hearing
people. Although important steps have been made towards the
development of real-world SLR systems, signer-independent
SLR is still one of the bottleneck problems of this research field.
In this regard, we propose a deep neural network along with an
adversarial training objective, specifically designed to address
the signer-independent problem. Specifically, the proposed
model consists of an encoder, mapping from input images to
latent representations, and two classifiers operating on these
underlying representations: (i) the sign-classifier, for
predicting the class/sign labels, and (ii) the signer-classifier, for
predicting their signer identities. During the learning stage, the
encoder is simultaneously trained to help the sign-classifier as
much as possible while trying to fool the signer-classifier. This
adversarial training procedure allows learning signer-invariant
latent representations that are in fact highly discriminative for
sign recognition. Experimental results demonstrate the
effectiveness of the proposed model and its capability of
dealing with the large inter-signer variations.
Index Terms—Sign language recognition, gesture
recognition, adversarial neural networks, deep learning.
The authors are with Centre for Telecommunications and Multimedia,
INESC TEC, 4200-465 Porto, Portugal, and also with Faculdade de
Engenharia da Universidade do Porto, 4200-465 Porto, Portugal, and also
with Faculdade de Ciências da Universidade do Porto, 4169-007 Porto,
Portugal, and also with Universidade Portucalense, 4200-072 Porto,
Portugal (Corresponding author: Pedro M. Ferreira, e-mail:
pmmf@inesctec.pt).
Cite: Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, and Jaime S. Cardoso, "Signer-Independent Sign Language Recognition with Adversarial Neural Networks," International Journal of Machine Learning and Computing vol. 11, no. 1, pp. 121-129, 2021.
Copyright © 2021 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).