Home // MMEDIA 2022, The Fourteenth International Conference on Advances in Multimedia // View article

Comparison of Two Approaches for Human Tense Situation Analysis in Car Cabin

Authors:
Quentin Portes
José Mendes-Carvalho
Julien Pinquier
Frédéric Lerasle

Keywords: Human interactions, multimodality, data fusion, audio and video features, end-to-end

Abstract:
The use of audio, video and text modalities to simultaneously analyze human interactions is a recent trend in the field of deep learning. The multimodality tends to create computationally expensive models. Our in-vehicle specific context requires recording a database to validate our approach. Twenty-two participants playing three different scenarios ("curious", "argued refusal" and "not argued refusal") of interactions between a driver and a passenger were recorded. We propose two different models to identify tense situations in a car cabin. One is based on an end-to-end approach and the other one is a hybrid model using handcrafted features for audio and video modalities. We obtain similar results (around 81% of balanced accuracy) with the two architectures but we highlight their complementary. We also provide details regarding the benefits of combining different sensor channels.

Pages: 1 to 8

Copyright: Copyright (c) IARIA, 2022

Publication date: April 24, 2022

Published in: conference

ISSN: 2308-4448

ISBN: 978-1-61208-942-3

Location: Barcelona, Spain

Dates: from April 24, 2022 to April 28, 2022