Sitemap in robots.txt

Posted By: glennh ()
Posted On: 2008-Mar-18 22:59

Google and Yahoo allow a Sitemap: <sitemap_location> line in the robots.txt.

Do any of the other search engines use that feature?
Does it this feature help?


Posted By: surrealillusions ()
Posted On: 2008-Mar-18 23:52

Might help the other search engines more. If you're already registed it with Google then Google will crawl it.

The xml sitemap shouldn't be linked to from the site, and so unviewable to normal users. So in theory, putting it in the .htaccess should be beneficial

smile


Posted By: Prowler (Staff)
Posted On: 2008-Mar-19 10:05

>> So in theory, putting it in the .htaccess should be beneficial ...

Putting a sitemap in .htaccess is pointless. .htaccess file is intended as a set of directives to the Apache server.

All search engines read up and cache your robots.txt. They specifically look for information about prohibited links and directories.




Posted By: surrealillusions ()
Posted On: 2008-Mar-19 13:20

why did i say .htaccess? i meant robots.txt wink

sorry

laugh :D