Bing Webmaster Tools began sending new Crawl Error Alerts to webmasters informing about issues “encountered by bing crawler during recent indexing of site(s).”
Here’s a quick rundown of the new alerts and issue types:
DNS Failures alert “deals with (Domain Name System “responsible for resolving hostnames to corresponding IP addresses”) issues, encountered when a crawler request a page from a server and the address of the hostname doesn’t get resolved,” informs bing. “This may indicate that the DNS service used for the website may be down or misconfigured.”
For example, “the Time-To-Live (TTL) setting for the DNS record may have been set to 0, which can lead to problems. In some cases, the DNS service used for a website may be actively blocking or filtering Bingbot – something you may be able to check using the Fetch as Bingbot tool,” bing explains.
Bing says, that despite the site-wide nature of DNS issues, they still report the number of attempts made to fetch specific URLs and how many of those attempts failed. After receiving the alert “we’ll upload a sample set of URLs the crawler tried to fetch when encountered the DNS issue to the Crawl Information tool, were they can be found in a brand new column called “DNS Issues”,” bing writes.
Connection Timeouts alerts highlights instances during which a web server refused a connection and or did not respond to crawler requests with a valid HTTP status in a timely fashion.
“Connection issues could indicate that your server was temporarily down or otherwise unable to respond. In some cases, it could mean that your server is actively refusing connections for Bingbot. In either case, some more investigation is needed,” bing stated. Ading, in this case also they upload a sample of URLs that couldn’t fetch due to Connection Errors to the Crawl Information tool, and these can be found under the new “Connection Timeouts column.”
Detailed 4xx and 5xx Server Response Alerts. Bing is also sending a number of granular alerts when a server responded to requests from Bingbot with HTTP status codes in the ranges 400-499 and 500-599.
“but know that we will only trigger alerts if we see a both a significantly large percentage as well as substantial increase in the amount of errors when compared to prior crawl cycles, so that you are made aware of potential spiking issues early on,” bing stated.
You can check out the List of Crawl Alerts for the full details.
Below is an example of an alert concerning a spike in 403 (Forbidden) responses, which indicates that the server may be unintentionally blocking valid Bingbot requests:
Anatomy of a Crawl Error Alert message contains “a description of the problem, the number of times bing saw the error type, as well as the percentage of all requests for which crawler saw this error during the most recent 24-hour crawl cycle.”
In addition, “the message contains a direct link to the relevant section in the Crawl Information tool (when a corresponding category exists) where you will find a sample set of URLs for which we encountered the issue.” bing added.
“Note that we upload these sample URLs after sending out all of our alerts (we want webmasters to know about spiking issues as soon as possible), so there may be a delay between you receiving the alert and us showing the additional data in the Crawl Information tool, but the sample URLs may help with any further investigation,” bing said.
Stay on Top of Your Alerts and Other Messages. Bing notes, that all alerts go directly to the Message Center inside Bing Webmaster Tools account and high-priority alerts also show up at the top of Webmaster tools dashboard.
However, “to stay on top of new, unread messages in your Message Center outside of the Webmaster Tools — including the new crawl alerts, but also index issues, and other critical notifications such as malware warnings — make sure to set up your Webmaster Tools profile to receive digests of your unread messages.”
You can do this by following the steps described in the help topic How to Set Email Alert Preferences. “Since alerts are more often the only way to find out about issues that can affect your performance in Bing (and as a consequence, Yahoo!), it’s recommend you set your frequency to “daily”,” bing added.