Energy, Memory, and Runtime Tradeoffs for Implementing Collective Communication Operations

Abstract

Collective operations are among the most important communication operations in shared- and distributed-memory parallel applications. In this paper, we analyze the tradeoffs between energy, memory, and runtime of different algorithms to implement such operations. We show that each existing algorithms have varying behavior and no algorithm exists that is optimal in all three regards. We also show examples where of three different algorithms solving the same problem, each algorithm is best in a different metric. We conclude by posing the challenge to explore the resulting tradeoffs in a more structured manner.