Printer Friendly Version
Email this thread to a friend
|
Featured Web Site Template |
|
Reflects user activity within the last 5 minutes
|
|
| Member |
Message |
solid71
Joined: Jun 15, 2007
# Posts: 49
|
Posted: 06/15/2007 03:09 am
Is it true that it is important to add sitemap to robots.txt now?
i.e...
User-agent: *
Disallow: /cgi-bin/
Disallow: /images/
Sitemap: [link]
I have a site in the top 3 in Google and it does not have the sitemap variable in the robots.txt.....by adding it, will it "mess me up"?
Thanks !!!
|
 |
g1smd
Moderator
Joined: Jul 28, 2002
# Posts: 10183
|
Posted: 06/15/2007 04:42 am
I don't trust XML sitemaps all that much.
I prefer to still use the plain HTML-page sitemap.
|
|
SportsGuy
Moderator
Joined: Aug 30, 2002
# Posts: 3593
|
Posted: 06/15/2007 04:47 am
A quick welcome!  answer below...
Here are some useful links to help you find your way around here at SEF:
The Search Function - this will help you find loads of useful things - just type in some keywords on the topic, select a date range (default is "for the last 2 weeks" and go. You can also search by author if you like.
The Main Index - this will show you the list of all the main sections here at SEF and get you quickly pointed to the space you're after.
Active Topics - this button can be found in the top navigation bar (as can all others) and is a list of today's active topics - very handy to see what's going on right now.
My Topics - this will show you a list of all the topics you've participated in - handy for finding that thread you posted in yesterday, quickly.
Our Forum Rules - worth reading through so you know the lay of the land here. We don't allow signatures or links in sigs. Link dropping is also frowned upon - we clean them up quickly, but best to read the rules first.
Finally, if you want to include a sample URL to illustrate a point, like www.domain.com/folder/file.html, be usre to wrap the URL in those little italics tags {i} - {/i} (replace the {} with []) and it'll show up. Otherwise, our system will try to make the link real on you. As a reminder, this is NOT an invitation to include a link to your site - it's strictly used to explain things like URL structures.
After all that, just another quick thanks for coming to hang with us. SEF is one of the oldest search engine optimization help forums online.
Enjoy.
The point of the robots.txt file is to keep robots from crawling a page. You WANT them to crawl the sitemap.
[ Message was edited by: Dinkar 06/15/2007 07:02 am ]
|
|
You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
- You have not yet logged in, or registered properly as a member
- You are a member, but no longer have posting rights.
- This is a private forum, for which you do not have permissions.
If you are a recent member, it's possible that you simply have not yet confirmed your account. Please
check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions
contained within.
If you cannot find this message, click here to Re-Send it.
|
If you are still experiencing problem, please read the
Login Assistance
Article for some advice on what may be causing your login not to work properly.
|
Switch to Advanced Editor and ...
Create a New Topic
or Reply to this Thread
|
|
