Communication and Timing Issues with MPI Virtualization
METADATA ONLY
Loading...
Author / Producer
Date
2020-09
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Computation–communication overlap and good load balance are features central to high performance of parallel programs. Unfortunately, achieving them with MPI requires considerably increasing the complexity of user code. Our work contributes to the alternative solution to this problem: using a virtualized MPI implementation. Virtualized MPI implementations diverge from traditional MPI implementations in that they map MPI processes to user-level threads instead of operating-system processes and launch more of them than there are CPU cores in the system. They are capable of providing automatic computation–communication overlap and load balance with little to no changes to pre-existing MPI user code. Our work has uncovered new insights into MPI virtualization: Two new kinds of timers are needed: an MPI-process timer and a CPU-core timer, the same discussion also applies to performance counters and the MPI profiling interface. We also observe an interplay between the degree of CPU oversubscription and the rendezvous communication protocol: we find that the intuitive expectation of only two MPI processes per CPU core being enough to achieve full computation–communication overlap is wrong for the rendezvous protocol—instead, three MPI processes per CPU core are required in that case. Our findings are expected to be applicable to all virtualized MPI implementations as well as to general tasking runtime systems.
Permanent link
Publication status
published
External links
Editor
Book title
EuroMPI/USA '20: 27th European MPI Users' Group Meeting
Journal / series
Volume
Pages / Article No.
11 - 20
Publisher
Association for Computing Machinery
Event
27th European MPI Users' Group Meeting (EuroMPI/USA '20)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03950 - Hoefler, Torsten / Hoefler, Torsten
Notes
Due to the Coronavirus (COVID-19) the conference was conducted virtually.
Funding
671697 - Mont-Blanc 3, European scalable and power efficient HPC platform based on low-power embedded technology (EC)
801039 - Exascale Programming Models for Heterogeneous Systems (EC)
801039 - Exascale Programming Models for Heterogeneous Systems (EC)