Google 'migration to better datastore' - Megastore replication

Google products run on multiple servers, multiple cells, and even multiple datacenters simultaneously, so that they keep running even if any one (or more) redundant underlying parts go down. The technology is called as “Multihoming,” a straightforward for read-only products like web search, but it's more difficult for products that allow users to read and write data in […]

Google products run on multiple servers, multiple cells, and even multiple datacenters simultaneously, so that they keep running even if any one (or more) redundant underlying parts go down. The technology is called as “Multihoming,” a straightforward for read-only products like web search, but it's more difficult for products that allow users to read and write data in real time, like GMail, Google Calendar, and App Engine. “This post describe how multihoming currently works in App Engine, and how we're going to improve it with a release next week. Google will be using a solution for consistency problem: Megastore replication, an internal library on top of Bigtable that supports declarative schemas, multi-row transactions, secondary indices, and recently, consistent replication across datacenters. Megastore replication was originally intended to replicate across multiple datacenters synchronously and atomically, using Paxos,” explained App Engine blog.