The Part 1 of this series of posts discussed the importance of reducing the number of URLs you expose through canonicalization. While you may have reduced the number of URLs you exposed to Live Search, a large site can still have a large surface area to crawl. In crawling your site, search engines may not get all the best content or can eat unnecessary bandwidth that you pay for. This is where HTTP compression and conditional GET can help.
Enabling HTTP compression: If your site is running Apache, you can leverage the mod_deflate tool. You can apply these filters site-wide or by selectively compressing only specific MIME types, determined by examining the header generated, either automatically by Apache or a CGI script or some other dynamic programming you create.
To enable compression for all MIME types, set the SetOutputFilter directive to a website or directory:
To enable compression on a specific MIME type (as in this example, “text/html”), use the AddOutputFilterByType directive:
AddOutputFilterByType DEFLATE text/html