Thin Serverless Functions with GraalVM Native Image

Abstract

Thin Serverless Functions with GraalVM Native Image

In the past two decades, cloud computing has reformed the IT service market. Serverless as its successor releases cloud users from heavy burdens of resources management and system administration. The auto-scaling and finer billing granularity make serverless easy-to-use and cost-efficient. However, serverless platforms are usually suffering the high latency caused by cold starts. Duplicated allocation of language runtimes, libraries and shareable program state for different invocations causes a huge waste of memory. A recent work, Photons, enables collocation of functions of the same user inside the same runtime. Photons reduce the memory footprint and the number of overall cold starts by a lot without performance degradation. However, Photons are implemented in the normal JVM, which still suffers from slow starts and a large memory footprint. All objects of concurrent invocations are located on the common object heap. Heavy burdens of garbage collection causes high response tail latency. In this project, we design and implement a serverless proxy runtime using GraalVM Native Image Isolate. Native Image Isolate supplies better isolation for collocated invocations. Separated heap spaces enable more efficient independent memory management, while state sharing among different isolates becomes difficult. Ahead-of-time compilation makes isolate proxy start up fast with low memory footprint. We propose isolate pooling and shared isolate to enable object caching and state sharing in our design. Different workloads are introduced to evaluate our isolate proxy. Isolate proxy can reduce runtime memory footprint by up to 59% and starts up 10x faster, with slightly a lower peak throughput compared to Photons. For memory-intensive workloads, distributed garbage collection reduces worst-case latency up to 50%. We use realistic serverless invocations pattern to perform a cluster-wide event-driven simulation, which shows that our isolate proxy can reduce the overall cluster memory usage by 30% while keeping slightly better overall response time compared to Photons. These results indicate that it is worthy to trade fast start up and low memory footprint for peak performance in serverless context.