PDA

View Full Version : Do I need a robots.txt file ???


K.Simmonds
08-27-2008, 06:57 AM
I have a few websites, and I have created a sitemap for them all. Do I need a robots.txt file as well? There is nothing at all I want to dissallow, I just want to make it as easy as possible for the search engines to find.

If I do need one, I believe that what I need is

User-agent: *
Disallow:


but where do I put it ? I'm not an expert, so please try to make it as easy to understand as you can. Thanks for any help.

edbr
08-27-2008, 07:34 AM
just save it in your home directory. As for need I believe it is liked by bots but not sure how important, However it will not harm to have one.

domedia
08-27-2008, 01:12 PM
It's only important to use if you want to stop the bots from going to certain section of your website. If you don't want the bots to be restricted in any ways, there's no need for a robots.txt