Home // International Journal On Advances in Software, volume 16, numbers 1 and 2, 2023 // View article
Hand Gesture Recognition System for the Physical Search System
Authors:
Shin Kajihara
Masato Okazaki
Chika Oshima
Koichi Nakayama
Keywords: Hand gesture; physical search system; MediaPipe
Abstract:
In this paper, we proposed a hand gesture recognition system for searching for lost objects using a physical search system (PSS). The PSS detects all displaced objects in a physical space using two cameras and a computer based on image differences-detection technology. When users tell the PSS what the lost object is, using hand gestures to describe it, such as its size and location, may be useful, as may words that describe the object's name, color, and the time it was last seen. The hand gesture recognition system was developed and experiments were conducted to examine how accurately the system can estimate the size indicated by the width between the user's hands. Also, to allow users to register various gestures as the commands they want to use, we investigated the recognition rate of finger gestures. As a result, the system could measure the width between users' hands with almost no errors, based only on the image taken by the camera and a marker. Moreover, the finger gestures could be recognized with high accuracy, unless it was difficult for users to reproduce the gestures that had been pre-registered. In the PSS, the displaced object's images are grouped into clusters that contain the same objects' images and data about their features. When a user tells the PSS the features of what they want to find, using their hand gestures, the PSS can present to the user images of the object in an appropriate folder (cluster) that matches the request. Finally, once the user identifies the lost object's image, the PSS displays where and when the object was last seen/lost.
Pages: 36 to 46
Copyright: Copyright (c) to authors, 2023. Used with permission.
Publication date: June 30, 2023
Published in: journal
ISSN: 1942-2628