A Platform-Agnostic Model and Benchmark Suite for Serverless Workflows

Abstract

In recent years, Serverless Computing has gained increasing attention in research and industry. Its potential in scalability and efficiency has led major cloud vendors to introduce workflow services that orchestrate serverless functions efficiently. However, these frameworks are based on entirely different architectures, whose characteristics have only poorly been studied. Moreover, the rapid development of these commercial systems makes it hard to keep track of their pros and cons. To fill the knowledge gap, we introduce a framework to compare and evaluate serverless workflow systems. It consists of three components: a model, a platform-agnostic workflow definition, and a benchmark suite. The model is a high-level abstraction of workflows and acts as the basis for a rigorous analysis. We introduce a new workflow definition that transcribes into multiple proprietary paradigms. We use it to implement the benchmark suite, composed of four micro-benchmarks and five application benchmarks. Together, they serve as a great tool to analyze in-depth the services offered by AWS, Azure, and Google Cloud. We evaluate them in terms of scalability, runtime, overhead, and more, yielding a great overview of the current state-of-the-art.