Strong consistency is not hard to get: Two-Phase Locking and Two-Phase Commit on Thousands of Cores
Author / Producer
Date
2019-09
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
Data
Abstract
Concurrency control is a cornerstone of distributed database engines and storage systems. In pursuit of scalability, a common assumption is that Two-Phase Locking (2PL) and Two-Phase Commit (2PC) are not viable solutions due to their communication overhead. Recent results, however, have hinted that 2PL and 2PC might not have such a bad performance. Nevertheless, there has been no attempt to actually measure how a state-of-the-art implementation of 2PL and 2PC would perform on modern hardware.
The goal of this paper is to establish a baseline for concurrency control mechanisms on thousands of cores connected through a low-latency network. We develop a distributed lock table supporting all the standard locking modes used in database engines. We focus on strong consistency in the form of strict serializability implemented through strict 2PL, but also explore read-committed and repeatable-read, two common isolation levels used in many systems. We do not leverage any known optimizations in the locking or commit parts of the protocols. The surprising result is that, for TPC-C, 2PL and 2PC can be made to scale to thousands of cores and hundreds of machines, reaching a throughput of over 21 million transactions per second with 9.5 million New Order operations per second. Since most existing relational database engines use some form of locking for implementing concurrency control, our findings provide a path for such systems to scale without having to significantly redesign transaction management. To achieve these results, our implementation relies on Remote Direct Memory Access (RDMA). Today, this technology is commonly available on both Infiniband as well as Ethernet networks, making the results valid across a wide range of systems and platforms, including database appliances, data centers, and cloud environments.
Permanent link
Publication status
published
External links
Book title
Journal / series
Volume
12 (13)
Pages / Article No.
2325 - 2338
Publisher
VLDB Endowment
Event
45th International Conference on Very Large Data Bases (VLDB 2019)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03506 - Alonso, Gustavo / Alonso, Gustavo
03950 - Hoefler, Torsten / Hoefler, Torsten
Notes
Funding
Related publications and datasets
Is supplemented by: https://doi.org/10.3929/ethz-b-000374738