Home // International Journal On Advances in Intelligent Systems, volume 13, numbers 1 and 2, 2020 // View article
Authors:
Sándor Darányi
Nasrine Olson
Eva Lindell
Nils-Krister Persson
Marina Riga
Efstratios Kontopoulos
Ioannis Kompatsiaris
Keywords: deafblind communication; haptograms; word and sentence semantics; ontology; smart textiles
Abstract:
By means of a proof-of-concept prototype, which is work in progress, we adopted a multidisciplinary approach to develop a smart-textile-based communication system for use by people with deafblindness. In this system, sensor technologies and computer vision are used to detect environmental cues such as presence of obstacles, faces, objects, etc. Focusing on the communication module here, a new ontology connects visual analytics with the user to label detected semantic content about objects, persons and situations for navigation and situational awareness. Such labelled content is then translated to a haptogram vocabulary with static vs. dynamic patterns, which are mapped to the body. A haptogram denotes a tactile symbol composed over a touchscreen, its dynamic nature referring to the act of writing or drawing. A vest made of smart textile, in the current variant equipped with a 4 x 4 grid of vibrotactile actuators, is used to transmit haptograms on the user’s back. Thereby system messages of different complexity -- both alerts and short sentences -- can be received by the user, who then has the option to respond by pre-coded questions and messages. By means of grids with more actuators, displays with higher resolution can be implemented and tested, paving the way for an extended haptogram vocabulary, covering more detailed ontology content.
Pages: 129 to 139
Copyright: Copyright (c) to authors, 2020. Used with permission.
Publication date: June 30, 2020
Published in: journal
ISSN: 1942-2679