Home // International Journal On Advances in Systems and Measurements, volume 13, numbers 3 and 4, 2020 // View article

Visual Customer Interaction through Emotion Detection and Face Landmarks

Authors:
Rui Duarte
Carlos Cunha
Valter Borges
André Ferreira
David Mota

Keywords: Image recognition; sentiment analysis; activity recognition; face landmarks; user satisfaction; retail environments

Abstract:
Understanding consumer behavior is a dynamic field, critically important to the success of companies and to consumer satisfaction. It is especially important in scenarios of intense competition, currently characteristic of the retail store industry, where companies fight for every individual customer. A great in-store experience encourages shoppers to become loyal customers, positive word of mouth and referrals. However, the opposite happens if customers’ needs are not met, a poor customer experience is provided and further visits of the customer may be at risk. Due to the dimension of several retail stores, a common problem is the location of products and the ability of customers to find them. When this occurs, sales decrease and customer satisfaction is not guaranteed, thus contributing to a poor customer experience. In this paper we present a method that targets user satisfaction, by providing the retail store a tool that detects if a product is not being found. Our approach is twofold: first, we detect if the customer is revealing signs of negative emotions by tracking the facial expressions, and second, the facial position of the customer is tracked to detect if he/she is repeatedly looking at the same place. In each context, a lost factor is updated and when a threshold is passed, the retail store assistant is notified for customer assistance. Results show that this method is well suited for emotion detection and will increase customer satisfaction and retail stores income.

Pages: 220 to 229

Copyright: Copyright (c) to authors, 2020. Used with permission.

Publication date: December 30, 2020

Published in: journal

ISSN: 1942-261x