diTii.com Digital News Hub

Sign up with your email address to be the first to know about latest news and more.

I agree to have my personal information transfered to MailChimp (more information)


Matt Cutts debunk Google Toolbar indexing

Google Matt Cutts has made a post to clear the mis-conception that the Google Toolbar led to Google indexing a page. To explain his points, he example, an InformationWeek article, which speculates how urls with the substring “mms2legacy” got indexed.

Matt disagree on the following content:

The reason for this [supposedly unlisted urls getting crawled –Matt], explained Ken Simpson, CEO of anti-spam company MailChannels, is that one’s Google Toolbar may be configured to pass URLs that one visits to Google for indexing. “If you run Google Toolbar, it knows pages you visit,” he said.

He says Google Toolbar doesn’t led to these urls being crawled, for e.g.:

the first result from the [inurl:mms2legacy] query given in the article. The first url in that result set that I saw was http://mediamessaging.o2.co.uk/mms2legacy/showMessage2.do?encMmsId=F1ABCF6D326A3F65 . Well, if you take the string F1ABCF6D326A3F65 from that url and search for that then you’ll find multiple references to that url. In the cases I looked into, we found these pages via someone publishing a link on http://my.opera.com or other places around the web. I can definitively say that all the urls I looked into were discovered via crawling regular old links.

Matt ends mentioning his 2006 post on the issue, explaining toolbar, and indexing. He also explain Why is Googlebot downloading information from our “secret” web server? issue with following advice to keep avoid your secret content being indexed by Googlebot:

Security through obscurity is not a great way to keep a url from being crawled. If you don’t want your content in Google’s web index then we provide a ton of advice on how to prevent that content from getting into Google.

Matt Cutts

Share This Story, Choose Your Platform!

Do NOT follow this link or you will be banned from the site!