More Virtual Promote ... Gazette · Webmaster & SEO Tools · Scumware.com · Free Website Templates

SEF

Search Engine Forums

Helping to make the Web - Since 1998

Virtual Promote Member Spotlight

Automate Link Exchange

Automate links mgt, DOWNLOAD now WHY pay monthly? (automate)

.
Forums	Index	Active Topics	New Topics	My Topics	Search	My Profile	Register	Inbox	Rules & TOS
.

Forum Index · Search Engine Forums · SEF Community & Networking · Professionals Corner · How far does a spider crawl down a page?

Add to hotlist

Mark This Forum Read

Printer Friendly Version

Email this thread to a friend

Moderator(s): yellowwing

Previous Topic Next Topic

Member	Message
saschaeh Joined: May 02, 2005 # Posts: 66	Posted: 05/12/2005 03:00 am I have generated a site map that pulls out spider friendly links to all the dynamic content in my DB. I have three categories with in each city and there are four cities. I have only pulled out one category of one city and it is already fairly long. How many full pages does a spider roughly go down. I read somewhere that spiders don’t always go all the way and that weighting is also lost the lower they go. 1: If this is true how far would you think that is? 2: I have made it user friendly and broken it down in a hierarchical fashion. Will i get penalized for this kind of thing? Thank you for your time! (Copy Shortcut)

g1smd Moderator Joined: Jul 28, 2002 # Posts: 10061 Website	Posted: 05/12/2005 05:03 am Google caches only the first 100kB of a page. For a short while, a few months ago, they did cache up to 250kB but are no longer doing so for new pages. The figure has reverted to 100kB again. For larger caches generated a few months back they are still retaining the larger page size for those only. How much is actually spidered and indexed is left for you to discover, simply by putting some unique "words" on your page and seeing which are indexed and which are ignored. (Copy Shortcut)

Vinnie Joined: Nov 03, 2003 # Posts: 66	Posted: 05/13/2005 07:59 am I may be have misunderstood what you are saying, but if I have not here is my take on it. Now this is coming from a brand new site released at the begining of last week with some real good inbound quality links to it. Google indexed the entire front page of 322 words. But having said that the entire site is built in XHTML and div tags, no tables no interuptions at all, clean stylish, optimised images, and readable. Now I am wondering if there had been tables etc if it would have done the same! I'm just waiting now until it makes it's damn mind up to get to the other pages in its own good time. (Copy Shortcut)

saschaeh Joined: May 02, 2005 # Posts: 66	Posted: 05/13/2005 12:29 pm Thanks! Ill def use your guys advice! Where did you find out about the 100K thing? Awesome tip on putting unique words at different levels of your page. All the best things! Sascha (Copy Shortcut)

g1smd Moderator Joined: Jul 28, 2002 # Posts: 10061 Website	Posted: 05/13/2005 01:12 pm The 100KB limit has been mentioned in many forums, and might also be found in the Google Guidelines for Webmasters, somewhere on their site. They did cache up to 250KB for a short while earlier in the year. The evidence was seen in the SERPs. They now only cache up to 100KB for new pages (as I found out when I briefly tested with a 120KB file last month). (Copy Shortcut)

Forum Index · Search Engine Forums · SEF Community & Networking · Professionals Corner · How far does a spider crawl down a page?

Who's Online?

There are 237 guests and 2 members in the forums right now.

los_angeles_seo, hungary

Reflects user activity within the last 5 minutes

Previous Topic Next Topic

You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:

You have not yet logged in, or registered properly as a member
You are a member, but no longer have posting rights.
This is a private forum, for which you do not have permissions.

If you are a recent member, it's possible that you simply have not yet confirmed your account. Please check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions contained within.

If you cannot find this message, click here to Re-Send it.

If you are still experiencing problem, please read the Login Assistance Article for some advice on what may be causing your login not to work properly.

Switch to Advanced Editor and ... Create a New Topic or Reply to this Thread

Related Forum Topics

340 sites +3 million page views per month (In: I Want to Sell My Website)
Page Design and Style (In: Graphic Design - Basic HTML, CSS & Usability)