Multiple sitemaps question

Member

Message

123_123

Joined: Apr 18, 2005

# Posts: 137

Posted: 05/02/2005 10:08 am

I have a site with around 30K HTML pages in 14 directories.
I have made a sitemap for each directory, 14 sitemaps

Now then what I have in effect is a jumbo list of URL's all listing pages which are unique content wise, no duplicates.

The question is:

Could I submit those seperate URls to the engines ? Ok of course I could submit them but is there any implications in doing so ?

How would you setup the sitemaps for the spiders to go spidering ?

By that I mean, would you put a sitemap link on the index page, that sitemap listing two more sitemaps, which in turn list a few more ? Would such a structure work from a spidering point of view, or what would be a better solution ?
Thanks

(Copy Shortcut)

g1smd

Moderator

Joined: Jul 28, 2002

# Posts: 10060

Website

Posted: 05/02/2005 10:24 am

Firstly, you don't submit sitemaps anywhere. The bots will find them if your site has incoming links from other sites and your sitemap is well linked within the site itself.

If your website has sections then you make one page of links for each section, and then have a main list which lists all of those. The main list should be linked from many important pages of the site.

Google recommends 100 links per page, and will not spider files larger than 100KB anyway. They did briefy spider pages up to, at least, 250KB but no longer do this.

(Copy Shortcut)

123_123

Joined: Apr 18, 2005

# Posts: 137

Posted: 05/02/2005 10:38 am

The site has 14 sections, but each, "section", could have as many as 2000 pages within it

I have created a sitemap for each of the 14 sections, but as you suggest idealy each map should really be no more than 100 links. Following that I think you are suggesting I break up the maps into handlable chunks.

Again following that, if say one section has 2000 pages, then do we make 20 maps to cover that section, and run sitemap1,2,3,4,5 etc links at the top of the index page for each section, or do we do a site map of the site maps ?

(Copy Shortcut)

g1smd

Moderator

Joined: Jul 28, 2002

# Posts: 10060

Website

Posted: 05/02/2005 11:02 am

If your website has sections then you make one page of links for each section, and then have a main list which lists all of those. The main list should be linked from many important pages of the site.

(Copy Shortcut)

linknz

Joined: May 22, 2005

# Posts: 10

Posted: 05/22/2005 02:48 am

My search engine uses a web spider called "Linknzbot" and that would have no problems with your directory structure and the number of files.
On the very first indexing of a web site the spider is controlled to only index six levels down picking up nine links per level but on the second visit it will go down fifty levels and collect all links found, it will index up to 500K per page as well.
The way that you describe your links shouldn't be a problem to most web spiders.
All the best
Dave A

(Copy Shortcut)

ratan_angel

Joined: Sep 05, 2005

# Posts: 9

Posted: 09/19/2005 02:44 am

On the very first indexing of a web site the spider is controlled to only index six levels down picking up nine links per level but on the second visit it will go down fifty levels and collect all links found, it will index up to 500K per page as well.

I was wondering if this is true for spiders of all Search Engines or does this apply to just a couple? If the latter is true, then which are the Search Engines to which this applies?

Apologies for the ignorance, but I reckon every bit of new information is an addition to one's knowledge.

Thanks guys!

Ratan

(Copy Shortcut)

yellowwing

Moderator

Joined: May 21, 2002

# Posts: 2524

Website

Posted: 09/24/2005 06:07 pm

What about the new Google site map [feed?] feature? Does anyone have the lowdown on that?

(Copy Shortcut)

You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:

You have not yet logged in, or registered properly as a member
You are a member, but no longer have posting rights.
This is a private forum, for which you do not have permissions.

If you are a recent member, it's possible that you simply have not yet confirmed your account. Please check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions contained within.

If you cannot find this message, click here to Re-Send it.

If you are still experiencing problem, please read the Login Assistance Article for some advice on what may be causing your login not to work properly.

Switch to Advanced Editor and ... Create a New Topic or Reply to this Thread

SEF