From the Blog

Learn About Robots.txt with Interactive Examples

Posted in SEO News & Updates

The Robot.txt belongs to the basic SEO best practise. Every website (at least 90%) should have a Robot.txt file.

The job of this file is to tell spiders like the Google bot which parts of the website is should access and which part it should not access. It enables the webmaster to keep parts of his/her website out of "Google's eyes".

Will Critchlow from Distilled wrote a great post on the SEOMoz blog explaining the Robot.txt file, what it does and how to use it on your website.

You can find the post over at:

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>