Windows Server 2008: Memory Management, Dynamic Kernel Addressing, Memory Priorities and I/O Handling

Today we will take a very brief look at some important enhancements to Memory Management, as well as talk about Dynamic Kernel Address Space, Memory Priorities and I/O Handling.  So let's dive right in ... The Memory Manager in Windows Server 2008 has been enhanced to provide better performance, scalability, security, and reliability in the […]

Today we will take a very brief look at some important enhancements to Memory Management, as well as talk about Dynamic Kernel Address Space, Memory Priorities and I/O Handling.  So let's dive right in ...

The Memory Manager in Windows Server 2008 has been enhanced to provide better performance, scalability, security, and reliability in the following areas:

  • Support for dynamic system address space, including on-demand allocation of system virtual address space and kernel page table pages, and support for very large registries.
  • Support for non-uniform memory architecture (NUMA) systems and systems with large pages, including additional device driver and Win32 NUMA API's.
  • I/O and section access improvements, including prefetch-style clustering for all types of page faults and system cache read-ahead (essentially pre-loading possibly-needed data into physical memory based on the current operation).
  • General performance improvements, including translation buffer optimizations and improvements to internal data structures and algorithmic performance.

Other than the new NUMA-related API's that developers of NUMA-aware applications or drivers need to be aware of, these management changes should be transparent to both applications and drivers.

We've done several posts on the memory address space on 32-bit systems.  There is a total of 4GB of virtual address space available (2^32 = 4GB) of which 2GB is allocated to the kernel by default, with the other 2GB being allocated to user-mode memory.  Internal components, device drivers, the file system cache, kernel stacks, per-session code data structures as well as paged and nonpaged pool memory are all mapped by the kernel.  Prior to Windows Vista and Windows Server 2008, the Memory Manager determined at boot time how much of this address space would be allocated to these different needs.  However, this led to situations whereby one area would become depleted, while another area had plenty of free space.  Exhaustion of one of these areas led to application and sometimes system failures.  To address this, the Dynamic Kernel Address Space feature, in 32-bit versions of Windows Server 2008, the Memory Manager dynamically manages the kernel's address space, allocating and deallocating space to various uses to meet the needs of the system.  As a result, the amount of virtual memory being used for paged and nonpaged pool, system PTE's and session space will grow and shrink based on system activity.  Thus, the need for manual tuning is reduced given the scaling improvements built into the operating system.

On all versions of Windows prior to Windows Vista, when a physical page (typically 4kb in size) that was owned by a process was reclaimed by the system, the Memory Manager would place the page at the end of the Standby List.  If the process wanted to access the page again, the Memory Manager would take the page from the Standby List and reassign it back to the process.  If a process wanted access to a new page of physical memory, then the Memory Manager would assign it the page that was at the front of the Standby List.  Thus all pages on the Standby List were treated as equals using only the order in which they were placed on the list to sort them.  Beginning with Windows Vista, each memory page has a priority ranging from 0 to 7.  The Standby List is divided into eight lists that each handle pages of a different priority.  When the Memory Manager wants to take a page from the Standby List, it takes pages from the low-priority lists first.  A page's priority is usually a reflection of the thread to which it was first allocated.  If the page is shared, then it reflects the highest of the memory priorities of the sharing threads.  Threads inherit their page-priority value from the process to which they belong.  By default, processes have a page-priority value of 5, however there are functions that allow applications and the system to change the process and thread page-priority values.  The real benefit to memory priorities comes into play when applications can take aware of the relative priorities of pages.  An example of this would be the SuperFetch feature in Windows Vista.

Full Article

Microsoft, WS2008, Windows Server 2008, Memeory, Management, Dynamic, Kernel, Memory Management, Memory Priorities, Architecture, Performance, Troubleshooting, Knowledgebase