Exchange Server 2007 "SP1 ESE Changes - Part 1"

Exchange 2007 Service Pack 1 introduces several changes in the Extensible Storage Engine (ESE). Two of these are: Removal of page dependencies Disablement of partial merges But first, in order to understand what is discussed below, we should have a brief discussion on ESE. ESE Architecture As you are well aware, operations that occur within […]

Exchange 2007 Service Pack 1 introduces several changes in the Extensible Storage Engine (ESE). Two of these are:

  • Removal of page dependencies
  • Disablement of partial merges

But first, in order to understand what is discussed below, we should have a brief discussion on ESE.

ESE Architecture

As you are well aware, operations that occur within Exchange are written to the transaction logs, changes are made to the database pages in memory, and then eventually those changes are flushed to the database file.

Thus, ESE utilizes a transactional architecture; specifically ESE follows the ACID methodology. ACID is an acronym and means:

  • Atomic - This means that a transaction state change is all or nothing.
  • Consistent - This means that a transaction must preserve the consistency of the data (i.e. transforming it from one consistent state to the next).
  • Isolated - This means that a transaction is a unit of operation and each assumes it is the only transaction occurring, even though multiple transactions occur at the same time.
  • Durable - This means that committed transactions are preserved in the database, even if the system stops responding.

The Extensible Storage Engine utilizes several components:

  • Transaction Logs - Transaction log files record the operations that are performed against the database, to ensure durability in case of system failure.
  • Checkpoint File - The checkpoint file is waypoint in the log sequence that indicates where in the log stream ESE needs to start a recovery in case of a failure.
  • Checkpoint Depth - The checkpoint depth is a threshold that defines when ESE begins to aggressively flush dirty pages to the database.
  • ESE Cache - An area of memory reserved to the Information Store process (Store.exe) that is used to store database pages in memory. By storing pages in memory, this can reduce read I/Os, especially when the page is accessed many times; in addition, this cache can be used in two ways to reduce write I/Os - by storing the dirty page in memory, there is the potential for multiple changes to be made to the page before it is flushed to disk; also, the engine can write multiple pages together (up to 1MB) using write coalescing.
  • Log Buffers - Operations performed against the database are first held in memory before they are written to transaction logs.
  • Version Store - An in-memory list of modifications that are made to the database. The version store is used for roll-back of transactions, for write-conflict detection, and for other tasks. The version store entry for a specific operation is cleaned up only one time when the following conditions are true:
    • The transaction that owns the operation has been either committed or rolled back.
    • All the transactions that are outstanding at the time that the operation was performed have been either committed or rolled back.
  • EDB File - Exchange database (.edb) files are the repository for mailbox (or public folder) data. Within the database, data is stored in 8 KB database pages, which in turn are stored in a collection of balanced trees, known as B+ trees. A B+ tree, is a multi-level index where all records are stored in leaf nodes, which are logically linked together to allow efficient searching and sorting of data. This allows a B+ tree to promote a persistent arrangement of data and to be able to locate and retrieve information from the disk quickly.

Note: For more information on B+ trees, please see http://en.wikipedia.org/wiki/B%2B_tree.

So how do the above components work together?

  1. An operation occurs against the database (e.g. client sends a new message), and the page that requires updating is read from the file and placed into the ESE cache (if it is not already in memory), while the log buffer is notified and records the operation in memory.
  2. The changes are recorded by the database engine but are not immediately written to disk; instead, these changes are held in the ESE cache and are known as "dirty" pages because they have not been committed to the database. The version store is used to keep track of these changes, thus ensuring isolation and consistency are maintained.
  3. As the database pages are changed, the log buffer is notified to commit the change, and the transaction is recorded in a transaction log file (which may or may not require a log roll and a start of a new log generation).
  4. Eventually the dirty database pages are flushed to the database file.
  5. The checkpoint is advanced.

For more information, please see Understanding the Exchange 2007 Store at
http://technet.microsoft.com/en-us/library/bb331958.aspx.

Full Article

Exchnage Server 2007, Service Pack, Features, Article