Home // BUSTECH 2020, The Tenth International Conference on Business Intelligence and Technology // View article

Challenge AI’s Mind: A Crowd System for Proactive AI Testing

Authors:
Siwei Fu
Xiaotong Liu
Anbang Xu
Rama Akkiraju

Keywords: Crowdsourcing; Artificial Intelligence; Proactive Testing.

Abstract:
Artificial Intelligence (AI) has burrowed into our lives in various aspects; however, without appropriate testing, deployed AI systems are often being criticized to fail in critical and embarrassing cases. In this paper, we propose the concept of proactive testing to dynamically generate testing data and evaluate the performance of AI systems. We further introduce Challenge.AI, a new crowd system that features the integration of crowdsourcing and machine learning techniques in the process of error generation, error validation, error categorization, and error analysis. The evaluation shows that the crowd workflow is more effective with the help of machine learning techniques.

Pages: 1 to 8

Copyright: Copyright (c) IARIA, 2020

Publication date: April 26, 2020

Published in: conference

ISSN: 2308-4391

ISBN: 978-1-61208-785-6

Location: Nice, France

Dates: from October 25, 2020 to October 29, 2020