Essential Business Server: Customer Experience Improvement Program (CEIP)

In our previous post, we discussed about the role of Windows Error Reporting (WER) in Essential Business Server (EBS). This topic will be discussing Customer Experience Improvement Program (CEIP) in EBS.  Various Microsoft applications ask for everyone's participation during setup or later in the program. A cursory examination of the legalese text sounds somewhat ominous.  "Send data […]

In our previous post, we discussed about the role of Windows Error Reporting (WER) in Essential Business Server (EBS). This topic will be discussing Customer Experience Improvement Program (CEIP) in EBS.  Various Microsoft applications ask for everyone's participation during setup or later in the program. A cursory examination of the legalese text sounds somewhat ominous.  "Send data to Microsoft? No way!" probably answer many users.  I hope that by the end of this post, you will understand exactly what kind of data we are collecting, and why we are collecting it.

Why do we want to collect data?

First of all, why do we have this program, and why do we want this data?

At Microsoft, we try our best to determine which features and UI design would best suit user needs.  We perform usability studies, conduct other research, and take in customer feedback.  However, the best feedback we can get on our products is from the actual users.

Have you ever hit an error dialog box that could have been avoided if only the UI were designed differently?  Have you ever wished you could tell the developers which features you use the most, and which ones you avoid because they are troublesome to use? That is why the CEIP program was developed at Microsoft and why we use it in EBS.

The Customer Experience Improvement Program (a mouth-full) is exactly what the name implies;  It's a program to improve our products (or "customer experience" as they call it) by collecting anonymous data about how our products are used by actual people.  We try to collect data (the kind of data is described later on) that helps us answer specific questions (called "Business Questions") that we can use to improve the next version of the product.

Here are some of the Business Questions we might try to answer about our EBS Admin Console:

  • What kinds of video resolutions do customers run at?
    • How many customers really run at the minimum resolution that we design the admin console for (800x600)?
    • How many customers use wide-screen resolutions?
    • What's the predominant video resolution used?
    • To what size do customers resize the Admin Console Window?  How many maximize the window?
  • How are customers using our product?
    • Do they use the Admin Console to add new users?
    • Which pages do they use most?  Which Wizards?
    • Which tasks do they run on each page?
  • What kinds of users are using our product?
    • Does more than one Administrator use the admin console at the same time?
    • Is EBS being used with many servers?  Or just our 3 servers?
    • Is EBS being used with how many users?

Knowing how our product is used helps us update the next release to meet the needs of real customers.

What kinds of data do we collect?

We collect many types of data, but it basically boils down to a few basic kinds:

  • Counters:  We count how many times something occurs.  How many servers are in the environment, etc.
  • Flags:  Has a specific error occurred?  Did the user cancel a wizard before completing?  DHCP in EBS enabled, or is the user using a different server/router?
  • Discrete Data:  Which of the 10 options did the customer pick?
  • Strings: Not used very often as there isn't much data that isn't private.  See below.

Full Article