Home // MMEDIA 2016, The Eighth International Conferences on Advances in Multimedia // View article

Towards Audio Enrichment through Images: A User Evaluation on Image Relevance with Spoken Content

Authors:
Danish Nadeem
Mariët Theune
Roeland Ordelman

Keywords: user evaluation study; linking audiovisual archives; multimedia semantics; audio augmentation

Abstract:
In a visual radio scenario, where radio broadcast is consumed on mobile devices (such as phones and tablets), watching pictures as you listen, may improve information or entertainment value of the programme. We assume that audio enrichment through images can be useful to users when the selection of images is semantically associated to the spoken content. In this paper, we report about a user study to evaluate the relevance of images selected automatically based on the speech content of audio fragments (audio interviews in the Dutch language). A total of 43 participants took part in the study. They listened to a set of audio fragments and performed an image rating task. In addition to that, we conducted a small follow-up study with 3 participants to shed more light on the results of the first study. We observed that merely keyword similarity between image captions and speech fragments may not be a good predictor for image relevance from a user viewpoint, and therefore we speculate that taking topic of speech into account may improve image relevance. Furthermore, from a user perspective on the added value of audio enrichment with images, we learned that the images should strengthen the understanding of audio content rather than distracting the listeners. The insights gained in the study will open room for further investigation of audio enrichment through images and its effect on user experience.

Pages: 44 to 47

Copyright: Copyright (c) IARIA, 2016

Publication date: February 21, 2016

Published in: conference

ISSN: 2308-4448

ISBN: 978-1-61208-452-7

Location: Lisbon, Portugal

Dates: from February 21, 2016 to February 25, 2016