Similarity Learning for CNN-Based ASL Alphabet Recognition

Fierro Radilla, Atoany Nazareth; Pérez-Daniel, Karina Ruby; Benítez-García, Gibran; Najera Garcia, Pedro; Fuentes Valdez, Ramona

doi:10.3233/FAIA210060

Similarity Learning for CNN-Based ASL Alphabet Recognition

Journal

Frontiers in Artificial Intelligence and Applications

New Trends in Intelligent Software Methodologies, Tools and Techniques

ISSN

0922-6389

1879-8314

Date Issued

2021

Author(s)

Fierro Radilla, Atoany Nazareth

Facultad de Ingeniería - CampCM

Pérez-Daniel, Karina Ruby

Facultad de Ingeniería - CampCM

Benítez-García, Gibran

Najera Garcia, Pedro

Facultad de Ingeniería - CampCM

Fuentes Valdez, Ramona

Facultad de Ingeniería - CampCM

Type

text::book::book part

DOI

10.3233/FAIA210060

URL

https://scripta.up.edu.mx/handle/20.500.12552/4333

Abstract

Sign language is an important communication way to convey information among the deaf community, and it is primarily used by people who have hearing or speech impairments. Besides, sign language represents a direct Human-Computer-Interaction (HCI) similar to voice commands therefore, the purpose of this study is to investigate and develop a system for American Sign Language (ASL) alphabet recognition using convolutional neural networks. Our proposal is based on semantic similarity learning using Siamese Convolutional Neural Network to reduce the intra-class variation and inter-class similarity among sign images in a Euclidean space the results of the siamese architecture applied to the ASL alphabet dataset outperform previous works found in the literature. From these results, using t-SNE visualization, we demonstrate that our hypothesis is correct; the ASL recognition improves when increasing the similarity among encoding of the images belonging to the same class and reducing it otherwise. ©2021 The authors and IOS Press. All rights reserved.