JBOD Hotmail's New Storage System - How Microsoft Supports Over One Billion Email Messages Each Day?

Curious to know how Microsoft stores the billions of Hotmail messages the company receive each day?In a September 22 blog post "A petabyte per week" - Chris Jones, SVP Windows Live, discuss how they address some of these challenges and reveal some major improvements that will be in place in company's storage system later this […]

Curious to know how Microsoft stores the billions of Hotmail messages the company receive each day?

In a September 22 blog post "A petabyte per week" - Chris Jones, SVP Windows Live, discuss how they address some of these challenges and reveal some major improvements that will be in place in company's storage system later this fall.

In the blog post, Kristof Roomp says, "Hotmail's storage system supports over one billion mailboxes and hundreds of petabytes of data (one petabyte is a million gigabytes, or a million billion bytes). The system services hundreds of thousands of simultaneous transactions from across the world. Just like the rest of Hotmail, our storage system is built using Microsoft technology, including Windows Server and Microsoft SQL Server. These systems are the backbone of Hotmail and are crucial to meeting the high standards we've set for the reliability and availability of our service."

Microsoft has been using a RAID set up for the Hotmail storage system for while.

He says that kind of storage system works if just one of the drives fails but " ... they don't help if the whole machine or the RAID controller runs into problems. For larger drives, we found that having completely independent copies (on hard drives not sharing the same machine or controller) was much more reliable than a significantly more expensive RAID configuration."

"Starting at the beginning of this year, we've been running the new system on a pilot cluster, using personal accounts of Microsoft employees who have volunteered to be test pilots. We've now finished certifying this new system, and are satisfied that it provides better reliability to users at a significantly lower price," said Roomp.

The new Hotmail storage system is name JBOD (Just a Bunch Of Disks), and ensures that the copies of data reside on independent hard drives, controllers, and machines. In essence, the hard drives, instead of being connected to one controller board, are handled in software. Roomp says, "The software we developed for the JBOD system monitors the hard drives schedules repair actions, detects failures, and diagnoses repairs. This software consists of a number of "watchdogs" that constantly monitor for certain types of failures. If the watchdog detects the failure that it is looking for, it raises an alert, which automatically triggers a repair process." About 30 million Hotmail users are already using the new system and 100 million more will be transferred in the next couple of months.

He says the new Hotmail storage system will use Flash Storage (also called SSD, or Solid State Drive) to handle features like "the list of messages in your inbox, read/unread status of your messages, conversation threading, mobile phone synchronization etc." This kind of data normally takes most of a hard drives activity. With SSDs handling this data and normal hard drives handling the actual email storage Roomp says, " ... we are able to take advantage of the trend in larger and cheaper hard drives without making any sacrifices in the performance of our system."

"The rollout has already begun and all new clusters that we deploy going forward will use JBOD. We will also retrofit JBOD to our existing systems over time. We have about 30 million users on JBOD today, with another 100 million moving to the system over the next couple months," added Roomp.