Elasticity in Cloud Databases and Their Query Processing

A central promise of cloud services is elastic, on-demand provisioning. For data-intensive services such as data management, growing and shrinking the set of nodes im-plies copying data to nodes with temporary membership in a service. The provisioning of data on temporarily available nodes is what makes elastic database services a hard prob-lem. At best, a node might retain (not destroy) its copy of the data while it provides an-other service; at worst, a node that rejoins the database service (or joins for the first time, or joins after a prior failure) requires a new copy of all its assigned data. The essential task that enables elastic data services is bringing a node and its data up-to-date. Strategies for high availability do not satisfy the need in this context because they bring nodes online and up-to-date by repeating history, e.g., by log shipping. We believe that nodes should become up-to-date and useful for query processing incremen-tally by key range. What is wanted is a technique such that in a newly added node, during each short period of time, an additional small key range becomes up-to-date, until even-tually the entire dataset becomes up-to-date and useful for query processing, with overall update performance comparable to a traditional high-availability strategy that carries the entire dataset forward without regard to key ranges. Even without the entire dataset being available, the node is productive and participates in query processing tasks. Our proposed solution relies on techniques from partitioned B-trees, adaptive merg-ing, deferred maintenance of secondary indexes and of materialized views, and query optimization using materialized views. The paper introduces a family of maintenance strategies for temporarily available copies, the space of possible query execution plans and their cost functions, as well as appropriate query optimization techniques.