FRIHOSTFORUMSSEARCHFAQTOSBLOGSCOMPETITIONS
You are invited to Log in or Register a free Frihost Account!


Google released robots.txt generator





the-guide
I've found an interested article from Google Webmaster Central Blog it's about Speaking the language of robots written by Sean Harding the Webmaster Tools Team on Thursday, March 27, 2008

They released and introduced you to your personal robot translator: the Robots.txt Generator in Webmaster Tools. It's designed to give you an easy and interactive way to build a robots.txt file. It can be as simple as entering the files and directories you don't want crawled by any robots and you can create fine-grained rules for specific robots and areas of your site as well if you need.

You can test your new robots.txt file with their robots.txt analysis tool after finished with the generator to see its effect. After you're done, just save your file to the top level (root) directory of your site, and you're good to go.

This new tool will helps you inform what you want to Googlebot and all other robots whose visit your site.

Hope this useful.
imagefree
thank you! Very Happy

Nice sharing
Peterssidan
Thanks for information! I hadn't noticed that. Even if it possible to do it without this tool it surely helps. Smile
the-guide
You're welcome, this will helps us write robots.txt precisely and easily, especially for one who has not familiarized it.
dude_xyx
thanks for the info. hadnt known it was there till i read this post.
Related topics
Robots.txt
robots.txt
Meta Tags
Robots.txt.... how important is it?
Frihost forum not crwaled by google?
Robots.txt
get to top in SEO ranking
about robots.txt
How to prepare a "robots.txt" to get crawled by Se
Please check this robots.txt file its getting complex for me
Evitar conteúdo duplicado com robots.txt
Question about robots.txt
Robots.txt
Robots.txt and Meta Robots
Reply to topic    Frihost Forum Index -> Webmaster and Internet -> SEO and Search Engines

FRIHOST HOME | FAQ | TOS | ABOUT US | CONTACT US | SITE MAP
© 2005-2011 Frihost, forums powered by phpBB.