Google Quality Raters: 'Stay at Home Rate Your Site without Seeing it'

PotPieGirl, who earlier this year in October found a copy of the version 3.18 March 30, 2011 of the Google Search Quality Rating Guidelines handbook used by Google Search Quality Raters around the world - has now come back with a new post on "Google Quality Raters. Who are they? What do they do?"She says […]

Google Quality Raters (Search Engine Optimization (SEO))PotPieGirl, who earlier this year in October found a copy of the version 3.18 March 30, 2011 of the Google Search Quality Rating Guidelines handbook used by Google Search Quality Raters around the world - has now come back with a new post on "Google Quality Raters. Who are they? What do they do?"

She says "Google Quality Raters are out there rating not only organic search results, but also Google ads (AdWords) and Videos, and probably more things but those are the three types of raters I am sure of."

In a nutshell, Google outsources this job to outside companies and those outside companies hire independent contractors to do the work.

The Google Quality Raters Handbook, a 125 page long, can be find linked in her post here.

In her November 17 blog post, she spcifically talks about the Raters that rate the organic results - called "Search Quality Raters."

She covers a lot of things about the Google Raters including:

  • How Google hires raters -
    "these people are "stay at home" type folks…moms, between jobs, students, etc," she said.

    They can apply for a Quality Rater position through one of these outside companies. If accepted, they then take a 2-part exam to qualify their ability.

    "The exam consists of two sections: Section 1 has 24 theory questions and Section 2 consists of 270 practical exams."

    If they pass the exams, they are then hired to work as an independent contractor (ie, work from home, no taxes taken out of pay - no employee benefits). They tend to work 10-20 hours per week and are paid by the hour. From what I can gather, they make something like $12 - $15 per hour.
  • What raters do
  • What their performance requirements are
  • and more…

She refrenced some fourm posts:

In one post a quality rater said they can rate the quality of the site without actually seeing the site, instead they can rate it based on the search result's snippet. One rater said in the forums, "When I was doing 20 I did a lot based just on snippets." Are you surprised?

In another post:

Your url can be judged simply by your snippet in the search results! It's possible no one actually LOOKS at your url at ALL!

Important Take-Away - Be SURE your snippet (ie, url meta description) is as relevant to your target query as possible. Now, Google can, and does, auto-generate these on their own MANY times, but try and control what you can.

Another quality rater posted this….which really bothers me:

"I would guess that for most people, the biggest challenge is remembering *precisely* what it is that you're rating. "

She also, revealed that their performance is judged by how many urls they do per hour. For the single url/keyword assignments, it appears they are required to do 30 per hour.

From that forum:

"Hi, I would like to ask you how many URLs per hour do you usually do? More than 60 or less than 60? I have no idea how many is the average because no one ever told me how many should I do. Thanks "

One other rater posted this as a response:

"I believe that 30 is the minimum expected for rating, within a few weeks after you're hired"

Another posted this:

"I believe that 30 is the minimum expected for rating, within a few weeks after you're hired (not including the test week). You'll get an email if you fall below standards. They stress quality over quantity, however, so that overall, a slower rater who rates really well may be preferable to a fast one who misses the mark more often. "May" is the operative word, we don't know what the criteria are for judging our work."

It really appears as none of them are actually SURE about what level they are expected to perform at. Interesting.

Another Quality Rater posted this:

"Experienced QRs, on average, how many U*Ls do you personally complete in one hour? I just got my 2nd progress report and productivity is only at 3 stars when I do about 45 to 50 per hour."

Quality Raters Guidelines are used by Google Quality Raters to aid them in classifying queries, measuring relevancy, and rating the search results. Brian Ussery in 2008 has discovered a revised copy of the Google Quality Raters Guidelines (April 2007).

Here are some search engine optimization guide contents:

Quality Rating Scale

  • Vital
  • Useful
  • Relevant
  • Not Relevant
  • Off-topic

Categories For Results That Can't Be Rated

  • Didn't Load
  • Foreign Language
  • Unratable

Spam Labels

  • Spam
  • Not Spam
  • Maybe Spam

Flags

  • Pornographic content
  • Malicious code on pages

Differentiating full e-commerce owners from affiliate marketers

Recognizing true merchants: Features that will help you determine if a website is a true merchant include:

* a "view your shopping cart" link that stays on the same site and updates when you add items to it,
* a return policy with a physical address,
* a shipping charge calculator,
* a "wish list" link, or a link to postpone purchase of an item until later,
* a way to track FedEx orders,
* a user forum, the ability to register or login,
* a gift registry, or
* an invitation to become an affiliate of that site

Please note the following:
Not all of the above need to be present for a merchant to be considered a true merchant. Yahoo! Stores are true merchants - they are not thin affiliates. Some true merchants will take you to another site to complete the transaction due to the fact that they utilize third party cart systems. Such merchants are not thin affiliates.