Home // AIMEDIA 2025, The First International Conference on AI-based Media Innovation // View article
Authors:
Fabian Engl
Timur Ezer
Juergen Mottok
Keywords: Machine Learning; Eye-Tracking; Usability; User Experience; UX
Abstract:
This paper compares six different Machine Learning (ML) algorithms — the k-nearest neighbor algorithm, a Support Vector Machine, a Multi-Layer Perceptron, a Random Forest, Gradient Boosting, and Adaptive Boosting — in their ability to classify users based on their usability and user experience (UX) ratings, using only eye-tracking data. A study was designed using three different websites from German drinking water providers, with the corresponding usability and UX ratings based on the User Experience Questionnaire (UEQ) and the AttrakDiff questionnaire. In total, 104 participants, contributing over 18 hours of eye-tracking data, took part in the study. The results indicate that Machine Learning models trained on smaller datasets, such as those in the field of eye-tracking, often achieve reasonable F1-scores without the need for extensive hyperparameter tuning. A comparison of random and Bayesian optimization approaches reveals that especially tree-based models benefit from Bayesian optimization. Among all models, the Support Vector Machine and Multi-Layer Perceptron perform the best, averaging F1-scores in the 90~% range, and demonstrating that usability and UX can be predicted using similar approaches across different websites within the same domain. Additionally, no significant difference was found between the usability and UX definitions of the UEQ and the AttrakDiff, suggesting that both are equally suitable for UUX predictions based on Machine Learning and eye-tracking.
Pages: 22 to 31
Copyright: Copyright (c) IARIA, 2025
Publication date: July 6, 2025
Published in: conference
ISBN: 978-1-68558-330-9
Location: Venice, Italy
Dates: from July 6, 2025 to July 10, 2025