Google Webmaster tools begin sending "Crawl Error" alerts in the form of messages in the Message Center to let webmasters know the state of their site.
"Since Googlebot regularly visits your site, we know when your site exhibits connectivity issues or suddenly spikes in pages returning HTTP error response codes (e.g. 404 File Not Found, 403 Forbidden, 503 Service Unavailable, etc). If your site is timing out or is exhibiting systemic errors when accessed by Googlebot, other visitors to your site might be having the same problem!," Google wrote.
The new Webmaster Tools Crawl Errors divided into two types: Site Errors and URL Errors.
Site Error alerts reprensts major site-wide problems rather than problems with specific pages. "These alert provides the number of errors Googlebot encountered crawling your site, the overall crawl error connection rate for your site, a link to the appropriate section of Webmaster Tools to examine the data more closely, and suggestions as to how to fix the problem," Google said.
Example of Site errors include:
- "Your DNS server is down or misconfigured.
- Your web server itself is firewalled off.
- Your web server is refusing connections from Googlebot.
- Your web server is overloaded, or down.
- Your site's robots.txt is inaccessible."
"If your site shows a 100% error rate in one of these categories, it likely means that your site is either down or misconfigured in some way. If your site has an error rate less than 100% in any of these categories, it could just indicate a transient condition, but it could also mean that your site is overloaded or improperly configured," Google explains.
URL Error alerts (pages that return a non-200 HTTP code, or incorrectly return an HTTP 200 code in the case of soft 404 errors) may occur on any well-configured site. Because, a a count of errors that indicates a serious problem for a small site might be entirely normal for a large site --that's why, Google says it only send alerts when a large spike in the number of errors are detected for any of the five categories: "Server error, Soft 404, Access denied, Not found or Not followed."
For example, "if your site routinely has 100 pages with 404 errors, we won't alert you if that number fluctuates minimally. However we might notify you when that count reaches a much higher number, say 500 or 1,000," Google explained.
Keep in mind that seeing 404 errors is not always bad, and can be a natural part of a healthy website.