Home // International Journal On Advances in Software, volume 8, numbers 3 and 4, 2015 // View article

Semantic Indexing based on Focus of Attention Extended by Weakly Supervised Learning

Authors:
Kimiaki Shirahama
Tadashi Matsumura
Marcin Grzegorzek
Kuniaki Uehara

Keywords: Semantic indexing, Focus of attention, Weakly supervised learning, Filtering

Abstract:
Semantic INdexing (SIN) is the task to detect concepts like Person and Car in video shots. One main obstacle in SIN is the abundant information contained in a shot where not only a target concept to be detected but also many other concepts are displayed. In consequence, the detection of the target concept is adversely affected by other irrelevant concepts. To overcome this, we enhance SIN based on a human brain mechanism to effectively select important regions in the shot. Specifically, SIN is integrated with Focus of Attention (FoA) which identifies salient regions that attract user's attention. The feature of a shot is extracted by weighting regions based on their saliencies, so as to suppress effects of irrelevant regions and emphasise the region of the target concept. In this integration, it is laborious to prepare salient region annotation that assists detecting salient regions most likely to contain the target concept. Thus, we extend FoA using Weakly Supervised Learning (WSL) to generate salient region annotation only from shots annotated with the presence or absence of the target concept. Moreover, rather than the target concept, other concepts are more salient in several shots. Features of these shots falsely emphasise concepts other than the target. Hence, we develop a filtering method to eliminate shots where the target concept is unlikely to be salient. Experimental results show the effectiveness for each of our contributions, that is, SIN using FoA, FoA extended by WSL, and filtering.

Pages: 410 to 419

Copyright: Copyright (c) to authors, 2015. Used with permission.

Publication date: December 30, 2015

Published in: journal

ISSN: 1942-2628