SEL is reporting about a Google's sting operation that it says proves Bing has been watching what people search for on Google, the sites they select from Google's results, then uses that information to improve Bing's own search listings. Bing doesn't deny this.
As a result of the apparent monitoring, Bing's relevancy is potentially improving (or getting worse) on the back of Google's own work. Google likens it to the digital equivalent of Bing leaning over during an exam and copying off of Google's test.
"I've spent my career in pursuit of a good search engine," says Amit Singhal, who oversees search engine's ranking algorithm. "I've got no problem with a competitor developing an innovative algorithm. But copying is not innovation, in my book."
To Sting A Bing
To verify its suspicions, Google set up a sting operation. For the first time in its history, Google crafted one-time code that would allow it to manually rank a page for a certain term (code that will soon be removed, as described further below). It then created about 100 of what it calls "synthetic" searches, queries that few people, if anyone, would ever enter into Google.
These searches returned no matches on Google or Bing -- or a tiny number of poor quality matches, in a few cases -- before the experiment went live. With the code enabled, Google placed a honeypot page to show up at the top of each synthetic search.
The only reason these pages appeared on Google was because Google forced them to be there. There was nothing that made them naturally relevant for these searches. If they started to appeared at Bing after Google, that would mean that Bing took Google's bait and copied its results.
This all happened in December. When the experiment was ready, about 20 Google engineers were told to run the test queries from laptops at home, using IE8, with Suggested Sites and the Bing Toolbar both enabled. They were told to enter the synthetic queries into the search box on the Google home page, and click on the results, i.e., the results we inserted. They started on Dec'17. By Dec'31, some of the results started appearing on Bing.
Here're some screenshots:
As we see it, this experiment confirms our suspicion that Bing is using some combination of:
- IE8, which can send data to Microsoft via its Suggested Sites feature
- the Bing Toolbar, which can send data via Microsoft's Customer Experience Improvement Program
or possibly some other means to send data to Bing on what people search for on Google and the Google search results they click. Those results from Google are then more likely to show up on Bing. Put another way, some Bing results increasingly look like an incomplete, stale version of Google results--a cheap imitation.
Here's the original response Microsoft's providing to who're asking:
"We use multiple signals and approaches in ranking search results. The overarching goal is to do a better job determining the intent of the search so we can provide the most relevant answer to a given query. Opt-in programs like the toolbar help us with clickstream data, one of many input signals we and other search engines use to help rank sites," said Stefan Weitz, Director, Bing.
Bing's formally responded how it uses more than 1,000 different "signals" to create its search algorithm. An excerpt of a blog post from Harry Shum, Bing cvp:
"To be clear, we learn from all of our customers. What we saw in today's story was a spy-novelesque stunt to generate extreme outliers in tail query ranking. It was a creative tactic by a competitor, and we'll take it as a back-handed compliment. But it doesn't accurately portray how we use opt-in customer data as one of many inputs to help improve our user experience."
In his comments today at the "Future of Search" event, Shum went a little further, saying
It's not like we actually copy anything. It's really about, we learn from the customers -- who actually willingly opt-in to share their data with us. Just like Google does. Just like other search engines do. It's where we actually learn from the customers, from what kind of queries they type -- we've query logs -- what kind of clicks they do. And let's not forget that the reason search worked, the reason web worked, is really about collective intelligence.
Shum added later: "We've been very clear. We use the customer data to help improve the search experience." And in the blog post, Shum suggests all search engines are, and should be, doing the same: "We all learn from our collective customers, and we all should."
[tags]sting,sting operation,web search,search results,spy,ranking signals,internet explore 8,ie8,ie7,internet explorer 7,ie9,internet explorer 9,farsight 2011,future of search,big think,harry shum,search quality[/tags]