Dynamic provisioning of storage resources: A case study with burst buffers
METADATA ONLY
Loading...
Author / Producer
Date
2020
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Complex applications and workflows needs are often exclusively expressed in terms of computational resources on HPC systems. In many cases, other resources like storage or network are not allocatable and are shared across the entire HPC system. By looking at the storage resources in particular, any workflow or application should be able to select both its preferred data manager and its required storage capability or capacity. To achieve such a goal, new mechanisms should be introduced. In this work, we present such a tool that dynamically provisions a data management system on top of storage devices. We propose a proof-of-concept that is able to deploy, on-demand, a parallel file-system across intermediate storage nodes on a Cray XC50 system. We show how this mechanism can be easily extended to support more data managers and any type of intermediate storage. Finally, we evaluate the performance of the provisioned storage system with a set of benchmarks. © 2020 IEEE.
Permanent link
Publication status
published
Editor
Book title
2020 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
Journal / series
Volume
Pages / Article No.
1027 - 1035
Publisher
IEEE
Event
34th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2020) (virtual)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Dynamic provisioning; Software-defined; Storage; Data manager
Organisational unit
Notes
Due to the Coronavirus (COVID-19) the conference was conducted virtually.