Printer Friendly Version
Email this thread to a friend
|
Featured Web Site Template |
|
Reflects user activity within the last 5 minutes
|
|
| Member |
Message |
g1smd
Moderator
Joined: Jul 28, 2002
# Posts: 10183
|
Posted: 07/26/2007 05:34 pm
The strange "hacking" case involving robots.txt, DMCA, The WayBack Machine and a specious lawsuit that could have had wide reaching implications had it succeeded...
http://www.theregister.co.uk/2007/07/26/wayback_firm_suit/
Thankfully it failed.
|
 |
david68
Joined: May 16, 2005
# Posts: 144
|
Posted: 07/27/2007 06:21 am
At least the wayback machine obeys robots.txt (I checked their site and my site is blocked as I disallowed all bots not specified as being allowed). Several bots don't obey it and it's really annoying. I end up blocking them through htaccess regardless on their motive just because their aren't being polite.
|
|
g1smd
Moderator
Joined: Jul 28, 2002
# Posts: 10183
|
Posted: 07/27/2007 12:57 pm
The point here is that for a short time, The WayBack Machine was not obeying robots.txt completely.
|
|
Prowler
Moderator
Joined: Aug 14, 2000
# Posts: 1721
|
Posted: 07/28/2007 01:18 am
>>The point here is that for a short time, The WayBack Machine was not obeying robots.txt completely.
For some technical reasons. Good Post g1smd. 
|
|
Curt
Joined: Eons Ago
# Posts: 3689
|
Posted: 08/12/2007 02:45 am
Wow! Guess all these bad bot spiders who don't play by the rules could find themselves in legal hot water some day for not obeying the robots.txt directives.
|
|
You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
- You have not yet logged in, or registered properly as a member
- You are a member, but no longer have posting rights.
- This is a private forum, for which you do not have permissions.
If you are a recent member, it's possible that you simply have not yet confirmed your account. Please
check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions
contained within.
If you cannot find this message, click here to Re-Send it.
|
If you are still experiencing problem, please read the
Login Assistance
Article for some advice on what may be causing your login not to work properly.
|
Switch to Advanced Editor and ...
Create a New Topic
or Reply to this Thread
|
|
