Home // International Journal On Advances in Software, volume 15, numbers 3 and 4, 2022 // View article

An In-depth Comparison of Experiment Tracking Tools for Machine Learning Applications

Authors:
Tim Budras
Maximilian Blanck
Tilman Berger
Andreas Schmidt

Keywords: Machine Learning; Experiment Tracking; Development Environment; MLOps

Abstract:
As the machine learning market is growing strongly and machine learning is increasingly being used productively, new challenges for developers and operators arise that haven’t been existing in traditional software development. One of these challenges is the versioning and reproducibility of models. To help solve this challenge experiment tracking tools exist, which keep track of the experimental development process of machine learning models. This paper describes the process of bringing a machine learning model to production and emphasizes its experimental nature and the challenges arising with it. Following the definition of a set of requirements for experiment tracking tools, 20 tools found in a market research are presented. Four of those tools are analysed in-depth, showing that differences between tools exist especially for advanced requirements. This paper also includes the progress the tools have made within the last year.

Pages: 152 to 164

Copyright: Copyright (c) to authors, 2022. Used with permission.

Publication date: December 31, 2022

Published in: journal

ISSN: 1942-2628