Continuing on tips to enbable AdSense crawler in Robots.txt for improved ad targeting. Google AdSense team suggest that the AdSense crawler be made to view all pages (including login paegs) that have AdSense code on them.
“Sometimes though, it can become problematic when a publisher has a members only site that requires the user to log in to view the site’s content. The AdSense ads that are located behind the login aren’t as relevant and targeted as they could be,” said Andrew Boni of Inside AdSense Team.
“If you’re a publisher with content behind a login page, be sure to set up an AdSense crawler login so that your ads are as well-targeted as possible. Doing so will give the AdSense crawler the ability to scan content on these pages, resulting in more relevant, targeted ads that can potentially increase your CTRs and revenue. Please note that adding a crawler login will only help to deliver better, more relevant ads to pages with AdSense code already on them, and will not have any effect on which pages of your site Google Search will index,” informs Boni.
“To grant the AdSense crawler access, log in to your AdSense account and click on ‘Account settings.’ From there, scroll down to the ‘Access and authorization’ section and click on ‘edit logins’ next to crawler logins. Then click on ‘Add a Login.’ You’ll be prompted to fill in pertinent information that the AdSense crawler will use to log in to your site and crawl your content,” Boni explained.
After that, you’ll need to log in to Webmaster Tools to verify your website, if you haven’t already done so. Once verified, the crawler will be able to log in successfully and scan your content to deliver more targeted ads.
For additional information, visit the help center article regarding displaying ads on login-protected pages.