Google officially stated the other day that they’ll be crawling through HTML forms. It’s an interesting move — their stated goal is to increase their coverage of the web by adding this new aspect to their crawling. I’ll note right away that this is not an immediate general addition to their crawling practices:
Only a small number of particularly useful sites receive this treatment, and our crawl agent, the ever-friendly Googlebot, always adheres to robots.txt, nofollow, and noindex directives. Google Webmaster Central Blog



0
Comments
Comments on this post