Home // INFOCOMP 2023, The Thirteenth International Conference on Advanced Communications and Computation // View article
Authors:
Hendrik Nolte
Julian Kunkel
Keywords: data management, high-performance comput- ing, provenance, reproducibility, IO performance.
Abstract:
Along with the increase in available compute power of high-performance computing (HPC) systems and the success of novel data-driven methods, the amount of data processed and the user groups increase as well. This gave rise to two big challenges: The traditional interaction scheme of users with modern HPC systems becomes more and more unsuited to deal with large data sets and many independent tasks working on these data sets. This highly manual way can quickly lead to unreproducible results and data loss due to missing backups since it is stored fragmented on multiple storage tiers. Similarly, domain-specific data management systems have been established to ease the burden of data and process management of particularly inexperienced users. These systems, however, only offer a very rigid, and tool-specific interaction scheme. This resulted in a gap between these two user groups, which even hinders large-scale cooperations across different domains. In this paper, we introduce the Governance-Centric interaction paradigm, a novel, and holistic concept which allows us to enforce data management plans to bridge this gap.
Pages: 13 to 20
Copyright: Copyright (c) IARIA, 2023
Publication date: June 26, 2023
Published in: conference
ISSN: 2308-3484
ISBN: 978-1-68558-073-5
Location: Nice, France
Dates: from June 26, 2023 to June 30, 2023