Home // MMEDIA 2014, The Sixth International Conferences on Advances in Multimedia // View article

A Semi-Automatic Multimodal Annotation Environment for Robot Sensor Data

Authors:
Konstantinos Tsiakas
Thedoros Giannakopoulos
Stasinos Konstantopoulos

Keywords: multimodal; annotation; tool; robot; MATLAB

Abstract:
In this paper, we present RoboMAE, a multi-modal sensor data annotation environment that allows humans to concentrate on high-level decisions producing full frame-by-frame annotations. Multi-modal annotation tools focus on interpreting a scene by annotating data on separate modalities. In this work, we focus on the cross-linking of the same object's recognition across the different modalities. Our approach is based on exploiting spatio-temporal co-occurrence to link the different projections of the same object in the various supported modalities and on automatically interpolating annotations between explicitly annotated frames. The backend automations interact with the visual environment in real time, providing annotators with immediate feedback for their actions. Our approach is demonstrated and evaluated on a dataset collected for the recognition and localization of conversing humans, an important task in human-robot interaction applications. Both the annotation environment and the conversation dataset are made publicly available.

Pages: 122 to 125

Copyright: Copyright (c) IARIA, 2014

Publication date: February 23, 2014

Published in: conference

ISSN: 2308-4448

ISBN: 978-1-61208-320-9

Location: Nice, France

Dates: from February 23, 2014 to February 27, 2014