High availability, elasticity, and strong consistency for massively parallel scans over relational data
Open access
Date
2014-08Type
- Journal Article
ETH Bibliography
yes
Altmetrics
Abstract
An elastic and highly available data store is a key component of many cloud applications. Existing data stores with strong consistency guarantees are designed and optimized for small updates, key-value access, and (if supported) small range queries over a predefined key column. This raises performance and availability problems for applications which inherently require large updates, non-key access, and large range queries. This paper presents a solution to these problems: Crescando/RB; a distributed, scan-based, main memory, relational data store (single table) with robust performance and high availability. The system addresses a real, large-scale industry use case: the Amadeus travel management system. This paper focuses on the distribution layer of Crescando/RB, the problem and theory behind it, the rationale underlying key design decisions, and the novel multicast protocol and replication framework it is composed of. Highlighting the key features of the distribution layer, we present experimental results showing that even under permanent node failures and large-scale data repartitioning, Crescando/RB remains fully available and capable of sustaining a heavy query and update load. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000075245Publication status
publishedExternal links
Journal / series
The VLDB JournalVolume
Pages / Article No.
Publisher
SpringerSubject
Scan-only query processing; Dynamic partial replication; High availability; Atomic multicastOrganisational unit
03506 - Alonso, Gustavo / Alonso, Gustavo
03689 - Kossmann, Donald (ehemalig)
More
Show all metadata
ETH Bibliography
yes
Altmetrics