Home // International Journal On Advances in Intelligent Systems, volume 17, numbers 1 and 2, 2024 // View article

Streamlining AI: Techniques for Efficient Machine Learning Model Selection

Authors:
Daniel Schönle
Christoph Reich
Djaffar Ould Abdeslam

Keywords: machine learning; nlp; efficiency; metric; soft- ware performance; automl

Abstract:
As machine learning (ML) systems become more ubiquitous, their resource requirements and associated financial burdens increase, highlighting the need to optimise energy consumption and costs to meet stakeholder expectations. While quality metrics for predictive ML models are well established, efficiency metrics are less commonly addressed. We present a comprehensive framework for evaluating efficiency metrics that facilitates the comparison of different types of efficiency. A novel efficiency metric approach (Compact Efficiency Metric) is proposed that considers resource usage, computational effort, and runtime in addition to prediction quality. Implementations for specific focus areas have been developed, such as the Quality-Focused Compact Efficiency Metric (QCO). This work also introduces a Pareto-based methodology for selecting ML models with an emphasis on efficiency. The QCO metric has undergone rigorous testing to validate its applicability, plausibility, and ability to adjust for variations in dataset size and hosting environment performance. This QCO metric has been applied to two different datasets and has been calculated for a wide range of ML models. In particular, when tasked with determining the optimal sequence length for transformer-based models, our metric identified an effective solution. The results from Pareto-based selection were congruent with those derived from the QCO metric, providing a viable approach for pre-selecting preferred solutions. This framework enables stakeholders to make informed decisions about the use and design of ML models, ensuring environmental responsibility and cost-effectiveness.

Pages: 73 to 87

Copyright: Copyright (c) to authors, 2024. Used with permission.

Publication date: June 30, 2024

Published in: journal

ISSN: 1942-2679