Home // CENTRIC 2023, The Sixteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services // View article

American Sign Language Recognition Using Convolutional Neural Networks

Authors:
Fatima-Zahrae El-Qoraychy
Yazan Mualla

Keywords: -Human-Computer Interaction; Convolutional Neural Networks; Sign Language Recognition

Abstract:
Sign Language Recognition (SLR) poses a challenge due to the rapid and intricately coordinated motions inherent in gestures. This research endeavors to address this complexity by leveraging Convolutional Neural Networks (CNNs). It presents a comprehensive exploration of diverse studies, methodologies, and inherent challenges in SLR, with a specific focus on harnessing CNN-based approaches for enhanced comprehension. At the core of this study lies a project aimed at the classification of American Sign Language gestures using CNN models rooted in the Visual Geometry Group 19 architecture. This initiative seeks to enrich the understanding and interpretation of manual gestures, fundamental to effective communication. Within this context, the article delves into pivotal aspects encompassing data diversification, model performance, and prospective limitations. Practical remedies are proposed, including data set augmentation and the incorporation of image masks, with the explicit objective of fortifying the precision and robustness of gesture recognition. For the validation and elucidation of classification outcomes, this study integrates the Gradient-weighted Class Activation Mapping (Grad-CAM) explanation model. This model uncovers salient regions within images, shedding light on the decision-making mechanisms of the CNN model, thereby enhancing transparency and comprehension.

Pages: 69 to 74

Copyright: Copyright (c) IARIA, 2023

Publication date: November 13, 2023

Published in: conference

ISSN: 2308-3492

ISBN: 978-1-68558-100-8

Location: Valencia, Spain

Dates: from November 13, 2023 to November 17, 2023