The SEOmoz blog has released their Crawl Test Tool to the public:
This tool is used to test how accessible your site is to search engines and can help you quickly diagnose potential crawling issues and give you an overview of your site's search friendliness. The tool will spider the URL you enter as well as all the internal links on that page (max 50 per report). For each spidered URL, it will examine the following: whether it's indexed in the major search engines, last time google spidered the page, http status code, primary keywords on the page, meta description, and the number of internal links on each page.
Since it’s introductary release, the new version has got two changes: The Crawl Test no longer uses the Yahoo! Term Extraction API to retrieve relevant keywords, instead it uses data from their in-house Term Targeting Tool.
Secondly, unless you are a Premium Member you can only run one report per day and each report will only spider 5 pages. Premium members can run as many reports as they like and it'll spider 50 pages.
Source:→ SEOmoz BlogSEOmoz, SEO, Search Engine, Tool, Crawl Test, Spider, Bot Crawl, Spider Crawl, Public, Release