Home // International Journal On Advances in Telecommunications, volume 13, numbers 3 and 4, 2020 // View article

Automatic Analysis of Nonverbal Mirroring Communication

Authors:
Oky Dicky Ardiansyah Prima
Yuta Ono
Kumiko Hosogoe
Miyu Nakano
Takashi Imabuchi

Keywords: mirroring communication; nonverbal communication; human pose estimation; DTW; cross-correlation.

Abstract:
Nonverbal communication plays an important role in social interaction. Mirroring, an action that mimics the nonverbal behavior patterns of their interaction partners, captures the attention of the Human-Computer Interaction (HCI) community. This action can help building rapport with others by making communication more effective and reflective. This study proposes a computer vision-based system that detects mirroring and analyzes the time lag during a face-to-face communication. Our approach consists of the following steps: (1) human pose estimation; (2) hand gestures quantization; (3) action detection based on Dynamic Time Warping (DTW); (4) estimation of mirroring time lag based on the cross-correlation. For this study, we recorded twenty face-to-face communication scenes using an omni-directional video camera with and without mirroring performed by the imitator. Results show that the DTW was able to detect actions having distinct gestures, whereas the cross-correlation was able to estimate the time lags for reactive mimicry of the imitator during the conversation.

Pages: 63 to 72

Copyright: Copyright (c) to authors, 2020. Used with permission.

Publication date: December 30, 2020

Published in: journal

ISSN: 1942-2601