robots.txt Creator, Generate Simple Robots Files

Enter The Files And Folders To Protect:

Enter each file name or folder path which you want to protect from all robots. This path is the path from the root of your server. If a file has a URL like www.your-domain.com/pages/hide-me/hidden.html then the path is /pages/hide-me/hidden.html

Sample file entry: "/logs/log-referr.html"

Sample folder entry (end a folder name with a "/" character): "/cgi-bin/"

Caution: If you enter just a "/" character this will block all spiders from indexing all files on your site!

Adding A Sitemap Tag To Your robots.txt File:

The last field allows you to enter the URL of your Sitemap if you have one on your site. This should be a complete URL like http://www.searchenginepromotionhelp.com/_sitemap-dynamic.xml.gz.

If you don't have a sitemap, generate your sitemap with our free online form.

robots.txt creator FAQ & Additional information

We hope you found this robots.txt creator helpful, please consider saying "thank you" with a link

We welcome your comments about this creator, please contact the webmaster.

Search Engine Promotion Help Site Map

© 2012 Search Engine Promotion Help
February 04 2012 03:19 UTC