Printer Friendly Version
Email this thread to a friend
|
how to get music url? (In: General Search Engine Optimization)
Featured Web Site Template |
|
Reflects user activity within the last 5 minutes
|
|
| Member |
Message |
SportsGuy
Staff
Joined: Aug 30, 2002
# Posts: 3600
|
Posted: 2007-Dec-29 14:11
Anyone have a list of "stop" characters or words lying around?
Those pesky little bits that stop spiders from crawling a URL further...
...or am I "old-school-thinking" here and we're passed all that these days?
Obviously making the URL too long is an issue, but otherwise, what stops spiders from crawling a URL fully?
|
 |
g1smd
Staff
Joined: Jul 28, 2002
# Posts: 10438
|
Posted: 2007-Dec-29 19:07
Can you elaborate more fully?
Especially the "crawl a URL" bit.
They either fetch a page or they don't. Part of that is Pagerank related and/or depends on the number of clicks away from the root it is.
|
|
SportsGuy
Staff
Joined: Aug 30, 2002
# Posts: 3600
|
Posted: 2007-Dec-30 12:57
I seem to recall that once upon a time, if you had certain characters or word sin your URL, the spider would stop crawling at that point - thus NOT indexing the page.
This could be completely out of date these days and I'm having a brain-fart, but I wanted to bounce the idea around.
Weren't the spiders having trouble at one point getting passed things like % signs in the URL itself, or spaces confused them or session ID info made them jump ship or something like that? I know you shouldn't use spaces when naming files and folders...
This could be old, out-of-date junk floating in my brain, but like I said, I wanted to float it for discussion.
|
|
pwcarguy
Joined: Jul 27, 2007
# Posts: 51
|
Posted: 2008-Jan-09 19:37
As late as last year, Google would ignore certain "ID" query strings. That appears to have changed.
http://www.mattcutts.com/blog/googlebot-keep-out/
|
|
Quadrille
Joined: Nov 15, 2000
# Posts: 1064
|
Posted: 2008-Jan-09 22:20
But that was just the fact that Google (and other SEs) found it hard to parse certain URLs; largely a thing of the past, but I understand that some strings with '?' are still affected.
I don't think stop words exist ... unless someone knows different?
|
|
SportsGuy
Staff
Joined: Aug 30, 2002
# Posts: 3600
|
Posted: 2008-Jan-10 01:15
You may well be bang on Quad - my thinking on it was sketchy, hence me asking. been a long time since I'd really thought about it, so I wanted to refresh on the point.
I recall at one point...(you know the rest)
Interesting that the "?" is still an issue...you'd think it would have been solved by now for crying out loud...just crawl the damn URL...
Thanks guys.
Duane
|
|
g1smd
Staff
Joined: Jul 28, 2002
# Posts: 10438
|
Posted: 2008-Jan-10 01:30
URLs with up to three parameters are fine as long as you use the same consistent parameter ordering, and you do not use session IDs in the URL. Beware of duplicate content issues at all times.
|
|
excell
Staff
Joined: Mar 19, 2001
# Posts: 14512
|
Posted: 2008-Jan-10 12:38
One has to ask - what sort of program are you using to generate the URLs? Are there not rewrite rules available within it?
|
|
SportsGuy
Staff
Joined: Aug 30, 2002
# Posts: 3600
|
Posted: 2008-Jan-10 20:33
One has to ask - what sort of program are you using to generate the URLs? Are there not rewrite rules available within it?
was this meant for me?
If so, my question is more theroetical than practical right now. Though I'm sure I'll now have a LOT of good reasons to use this info... 
|
|
You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
- You have not yet logged in, or registered properly as a member
- You are a member, but no longer have posting rights.
- This is a private forum, for which you do not have permissions.
If you are a recent member, it's possible that you simply have not yet confirmed your account. Please
check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions
contained within.
If you cannot find this message, click here to Re-Send it.
|
If you are still experiencing problem, please read the
Login Assistance
Article for some advice on what may be causing your login not to work properly.
|
Switch to Advanced Editor and ...
Create a New Topic
or Reply to this Thread
|
|
Inbox
