Google Search team published a video of the "uncut" discussion of a proposed algorithm change (an upcoming change to spell correction system). "The video footage was captured on December 1, 2011 at Google's weekly "Quality Launch Review" meeting in Mountain View, and some participants joined on videoconference from Google's remote offices around the globe, including our offices in Moscow, New York, Zurich, Seoul, Haifa and Tokyo," blogged Google.
For those, who may find, the language "technical," annotations are provided to some context for the discussion.
Google highlight a few things that you may notice in the video:
- "The specific change discussed in this video improves spelling suggestions for searches with more than 10 words and it impacts only .1% of our traffic. Still, you can see the scrutiny and thoughtfulness that goes into approving this change.
- Every change has a dedicated search quality analyst assigned to study the impact. This analyst is not part of the engineering team building the change, but instead offers a separate opinion on whether the change is good for users.
- The search team relies heavily on the results of experimental data to make decisions. During the meeting, we rely on detailed analyst reports including the results of click evaluations and side-by-side experiments. These reports can sometimes be more than 25 pages long.
- Launch reports include specific examples to illustrate broader trends in the data. Rather than manually change one example, our engineers look for algorithmic ways to improve millions of queries.
- Search algorithm improvements often rely on and impact many different systems, so engineers with expertise in different areas all need to come together to make the best decision for the user, balancing all the tradeoffs involved (relevance, spam, latency, cost, language impact, etc.)," Google said.
Check out the video below: