I am assuming you already use a robots.txt file if not go to robotstxt.org because you need to know this stuff.

Anyway, to add your sitemap to your robots.txt file just add the sitemap line to the end of the file after all your disallow commands like so:

User-agent: *
Disallow: /somefolder/
Disallow: /somethingelse/
Sitemap: http://www.mysite.com/sitemap.php

Now next time the search engines crawl your site they will automatically find your sitemap.

If you have a new site you will should probably still submit the sitemap URL to the search engines through their webmaster interfaces or use a ping.

Submit Sitemap to Google or Ping Google with your Sitemap

Submit Sitemap to Yahoo or Ping Yahoo with your Sitemap

Submit Sitemap to MSN Live.com

Submit Sitemap to Ask.com or Ping Ask.com by hitting this address:
http://submissions.ask.com/ping?sitemap=http://www.yourdomain.com/sitemap.xml

One Response to “Don’t just ping your sitemap – add it to your robots.txt file”
  1. rep mgmt says:

    do you know of a script that dose this for you?