Home // MOBILITY 2011, The First International Conference on Mobile Services, Resources, and Users // View article

Real-time Cognitive-Capacity-Sensitive Multimodal Information Exchange for the Cockpit Environment

Authors:
Atta Badii
Ali Khan

Keywords: multimodal command and control; cognitive-load; minimal distraction; cognitive control; mixed initiative taking; man-in-the-loop; machine learning

Abstract:
Deployment of multimodal interfaces in an environment of high cognitive-load, e.g., the cockpit environment of a police helicopter or motorcyclist in-pursuit, using audio communication with the need to frequently change radio channel or mode/focus of attention under time-constraints, whilst multi-tasking, can impose an extra load on the driver in addition to that due to the complexity of vehicle manoeuvring. This can raise concerns regarding driver safety. There is a need for a multi-sensor, multimodal information exchange interface that is minimally distracting by design, with consideration for the cognitive capacity of the driver at any given point in time. This paper presents the MoveON Jacket prototype with a Cognitive Capacity Respecting Multimodal Information Exchange framework that offers a minimal-distraction interface to motorcyclists enabling access to information online in real-time while supporting enhanced compliance with road safety constraints. This project has delivered the first validated implementation of an architecture for: a) minimal-distraction, b) cognitive capacity awareness, c) multimodal communication and control, d) graceful in-line man-in-the-loop cognitive control integration, e) performance improvement context caching to support an off-line learning capability for incrementally optimised man-machine mixed-initiative taking, f) performance enhancement and personalised ambient driver’s pal adaptation.

Pages: 170 to 176

Copyright: Copyright (c) IARIA, 2011

Publication date: October 23, 2011

Published in: conference

ISSN: 2308-3468

ISBN: 978-1-61208-164-9

Location: Barcelona, Spain

Dates: from October 23, 2011 to October 29, 2011