Earlier this week, we reported that according to Hitwise "Bing had its highest search success rate 81.54% in January, while Google's was 65.58%. Hitwise defines a successful search as one that results in a visit to a website from a search engine's result page.
Google's Matt Cutts is skeptical about the latest Hitwise "search success rate" data that showed Bing provided users with more accurate results than Google. Last night, on Google Buzz, Cutts voiced his objection to Hitwise "search success rate" by reposting an old 2009 comment:
It sounds like Hitwise's definition is "A successful search is defined as one where the consumer leaves the search engine after performing a search." In another words, the user does a query and then goes somewhere else. That doesn't sound the same as success to me; it just sounds like leaving the site.
Are you able to determine whether the user clicked on a search result vs. just left the search engine to go to another site? There's a difference between an abandoned search and clicking on a search result, but both result in the user searching and then going to a different site. By Hitwise's definition, wouldn't doing a query on Bing and then going to Google or Yahoo count as a "successful search" on Bing? I'm also assuming that you can't measure if the user got the information that they needed from the search results without needing to click to another site.
I think the phrase "successful search" is considerably less accurate than "left the site after searching," because someone can leave a site for lots of different reasons.
Cutts in the end says, "Hitwise later confirmed to me that they don't know whether the user actually clicked on a search result or just went to a completely unrelated site." Given all that, I'm surprised to see Hitwise is still pushing this metric and still calling it "search success rate." I'm also surprised that Computerworld turned that into "Bing searches more accurate than Google's," Cutts said.