Home // CENTRIC 2020, The Thirteenth International Conference on Advances in Human-oriented and Personalized Mechanisms, Technologies, and Services // View article
Authors:
Stephan Böhm
Judith Drebert
Sebastian Meurer
Keywords: chatbot; Wizard-of-Oz testing; prototyping; chatbot development; recruiting
Abstract:
Chatbots can be utilized to automate various business processes to add value for companies and users – for example, in the form of efficiency enhancement. Throughout the process of chatbot development, the integration of user feedback within a user-centered conversational design process is essential. In our study, we investigated chatbots in recruiting, a field within human resource management that is characterized by a high proportion of repetitive and standardized tasks. This pre-study applies a Wizard-of-Oz approach in which a basic dialog concept is tested in a very early phase of the project, simulating the chatbot functionality by a human operator. In this way, valuable user feedback on the general suitability of the dialog design can be gathered without coding chatbot functionalities. In total, eight users participated in our 60-minute experiment to conceptionally validate our idea and test the simulated Frequently Asked Questions (FAQ) chatbot. The research brought important insights into the basic concept and allowed us to collect new user intents not considered in the design. As a result, the tested concept proved to be suitable and of value for the users. Despite relatively long response times, only one participant suspected that they were not interacting with a chatbot but a human operator. The feedback on the user satisfaction with the completeness of the predefined answers and competence setup of the simulated chatbot was indifferent and rather moderate. However, most of the participants considered the tested scenario as relevant and stated a high user value for implementing the proposed chatbot in a recruiting process. Moreover, the Wizard-of-Oz approach generated appropriate input for improving the chatbot concept (e.g., intents, entities, criteria for satisfaction and acceptance enhancement) and valuable practical insights for developing a recruiting FAQ chatbot aligned to user needs.
Pages: 48 to 56
Copyright: Copyright (c) IARIA, 2020
Publication date: October 18, 2020
Published in: conference
ISSN: 2308-3492
ISBN: 978-1-61208-829-7
Location: Porto, Portugal
Dates: from October 18, 2020 to October 22, 2020