Supporting persistent metadata for scalable cloud services

Abstract

In this project we addressed the problem of providing a persistent, redundant storage system with high availability. This problem needs to be efficiently solved in order to provide support for storing metadata of a Platform as a Service (PaaS) solution in a cloud computing system. The developed solution is very compact, features an adjustable level of redundancy to achieve the desired level of availability, and delivers very high performance. Roughly speaking, it is able to service approximately 10,000 requests per second on current server platforms. Additionally, the response time is mostly determined by the commit time, which is a configurable parameter. When immediate commit is implemented, response time is as short as 10 ms for unloaded systems. The additional delay in heavily loaded systems never exceeded 15 ms. The implemented solution is based on Raft and employs LevelDB as the underlying database. However, several enhancements have been introduced to optimize performance and consistency. This includes guaranteeing that data - dependent operations can be sequentially issued to database while ensuring consistency without waiting for the request callback; and enabling the Raft node attached to a Raft client interface that issued a given operation to be the one that returns the result of such an operation; and moving from a five step approach to a three step approach for cluster membership reconfiguration.