Add To Favorites | St George Blog
Sales: 888.55.WEBSITE ext. 1
Support: 888.55.WEBSITE ext. 2

Robot File Help

To block Google's Imagebot from indexing images on your website add to your robots.txt file:

User-agent: Googlebot-Image
Disallow: /

To block only Google from crawling gif image files insert into your robots.txt file:

User-agent: Googlebot
Disallow: /*.gif$

To block only Google from crawling dynamic pages put into your robots.txt file:

User-agent: Googlebot
Disallow: /*?

To block ALL bots from crawling the pages under directory no-crawl put into your robots.txt file:

User-agent: *
Disallow: /no-crawl/

To block only Google from crawling pages under directory private add to your robots.txt file:

User-agent: Googlebot
Disallow: /private/

You can tell the bots to index a page, but not to follow the outbound links you'd use the following tag:

<META NAME="ROBOTS" CONTENT="NOFOLLOW">

You can prevent only Google from indexing a page by adding the following meta tag into the<HEAD> portion of THAT PAGE:

<META NAME="GOOGLEBOT" CONTENT="NOINDEX, NOFOLLOW">

You can place the following meta tag into the <head> portion of THAT PAGE to prevent all bots from indexing it:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">