Home > Archive > 2021 > Volume 11 Number 2 (Mar. 2021) >
IJMLC 2021 Vol.11(2): 121-129 ISSN: 2010-3700
DOI: 10.18178/ijmlc.2021.11.2.1024

Signer-Independent Sign Language Recognition with Adversarial Neural Networks

Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, and Jaime S. Cardoso

Abstract—Sign Language Recognition (SLR) has become an appealing topic in modern societies because such technology can ideally be used to bridge the gap between deaf and hearing people. Although important steps have been made towards the development of real-world SLR systems, signer-independent SLR is still one of the bottleneck problems of this research field. In this regard, we propose a deep neural network along with an adversarial training objective, specifically designed to address the signer-independent problem. Specifically, the proposed model consists of an encoder, mapping from input images to latent representations, and two classifiers operating on these underlying representations: (i) the sign-classifier, for predicting the class/sign labels, and (ii) the signer-classifier, for predicting their signer identities. During the learning stage, the encoder is simultaneously trained to help the sign-classifier as much as possible while trying to fool the signer-classifier. This adversarial training procedure allows learning signer-invariant latent representations that are in fact highly discriminative for sign recognition. Experimental results demonstrate the effectiveness of the proposed model and its capability of dealing with the large inter-signer variations.

Index Terms—Sign language recognition, gesture recognition, adversarial neural networks, deep learning.

The authors are with Centre for Telecommunications and Multimedia, INESC TEC, 4200-465 Porto, Portugal, and also with Faculdade de Engenharia da Universidade do Porto, 4200-465 Porto, Portugal, and also with Faculdade de Ciências da Universidade do Porto, 4169-007 Porto, Portugal, and also with Universidade Portucalense, 4200-072 Porto, Portugal (Corresponding author: Pedro M. Ferreira, e-mail: pmmf@inesctec.pt).


Cite: Pedro M. Ferreira, Diogo Pernes, Ana Rebelo, and Jaime S. Cardoso, "Signer-Independent Sign Language Recognition with Adversarial Neural Networks," International Journal of Machine Learning and Computing vol. 11, no. 1, pp. 121-129, 2021.

Copyright © 2021 by the authors. This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).

General Information

  • ISSN: 2010-3700 (Online)
  • Abbreviated Title: Int. J. Mach. Learn. Comput.
  • Frequency: Bimonthly
  • DOI: 10.18178/IJMLC
  • Editor-in-Chief: Dr. Lin Huang
  • Executive Editor:  Ms. Cherry L. Chen
  • Abstracing/Indexing: Inspec (IET), Google Scholar, Crossref, ProQuest, Electronic Journals Library.
  • E-mail: ijmlc@ejournal.net

Article Metrics