diTii.com Digital News Hub

Sign up with your email address to be the first to know about latest news and more.

I agree to have my personal information transfered to MailChimp (more information)

Oct042009

“Human evaluators are pretty important for us today” Google Scott Huffman

Scott Huffman runs the evaluation team at Google explained how Google’s trying to stay ahead in search, and runs all those experiments,which include human evaluators in addition to Google’s massive computer infrastructure. “human evaluators are pretty important for us today. The more automated or user behavior/click-based things really give you complementary kinds of data. Both have noise in them: Human evaluators make mistakes. Clicks are hard to interpret; people click or don’t click for all kinds of reasons. The clicks obviously tell you what users are actually doing, and you get them at volume, at a real scale, but it’s hard to interpret. Human evaluators, there’s noise in terms of mistakes, but we can go deeper on specific examples. For this pretty rare query, we can generate examples that our ranking engineers can go look at. Where we don’t get a correlation, that’s a big red flag,” said Huffman.

Share This Story, Choose Your Platform!