There are many elements and many tradeoffs involved in achieving performance that meets everyone’s expectations. We’ve re-dedicated ourselves to work in this area in Windows 7 and IE8 Beta 2 [IE8 Performance] . This is a major initiative across each of our feature teams as well as the primary mission of one of our feature teams (Fundamentals).
“We have criteria that we apply at the end of our milestones and before we go to beta and we won’t ship without broadly meeting these criteria," Sinofsky stated. "Sometimes these criteria are micro-benchmarks (page faults, processor utilization, working set, gamer frame rates) and other times they are more scenario based and measure time to complete a task (clock time, mouse clicks). We do these measurements on a variety of hardware platforms (32-bit or 64-bit; 1, 2, 4GB of RAM; 5400 to 7200 RPM or solid-state disks; a variety of processors, etc.) Because of the inherent tradeoffs in some architectural approaches, we often introduce conditional code that depends on the type of hardware on which Windows is running.”
Performance is made up of many different elements. We could be talking about response time to a specific request. It might mean how much RAM is “typical” or what CPU customers need. We could be talking about the clock time to launch a program. It could mean boot or standby/resume. It could mean watching CPU activity or disk I/O activity (or lack disk activity). It could mean battery life. It might even mean something as mundane as typical disk footprint after installation. All of these are measures of performance. All of these are systematically tracked during the course of development. We track performance by running a known set of scenarios (there are thousands of these) and developers can run specific scenarios based on exercising more depth or breadth. The following represent some (this is just a partial list) of the metrics we are tracking and while developing Windows 7.