In addition to crawling & indexing web pages via submitted URLs, Google recently launched a feature that uses RSS and Atom feeds to discover new web pages. This helps Google index new webpages faster than traditional methods. As a result, you must make sure that crawling of you feeds isn’t disallowed by your robots.txt. To find out if Googlebot can crawl your feeds, test your feed URLs with robots.txt tester. Going forward, we might also explore mechanisms such as PubSubHubbub to identify updated items, explained Google.
By Deepak Gupta Posted on ,
Share This Story, Choose Your Platform!
About the Author:
Deepak Gupta is a IT & Web Consultant. He is the founder and CEO of diTii.com & DIT Technologies, where he's engaged in providing Technology Consultancy, Design and Development of Desktop, Web and Mobile applications using various tools and softwares. Sign-up for the Email for daily updates. Google+ Profile.
WebSync for Microsoft stack (IIS/MSSQL/.NET)
Google plans 'host you own server' for federation
@diTii Facebook Page
Recent News Topics