Prediction and Interpretability of HPC I/O Resources Usage with Machine Learning
Résumé
I/O management tools proposed for high-performance computing (HPC) environments usually rely on accurate I/O bandwidth estimations to make their decisions. However, finding the correct I/O subsystem configuration that provides this maximal I/O bandwidth is particularly hard. In this work, we focus on finding a good estimate of the number of I/O resources (e.g., OSTs and I/O nodes) that provides the maximal bandwidth while minimizing the system occupation and taking into account the natural I/O variability. We use machine learning techniques to do so, focusing on intrinsic application features and system configurations. We show I/O resource usage is predictable and further study the impact of different features. We also validate our models with four I/O kernels from real applications. Finally, we show that our model, when used for resource allocation, can improve application performance.
Origine | Fichiers produits par l'(les) auteur(s) |
---|