Printer Friendly Version
Email this thread to a friend
|
Featured Web Site Template |
|
Reflects user activity within the last 5 minutes
|
|
| Member |
Message |
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-09 04:23
In answer to your question, "unreviewed", at the moment Cloaker allows you to individually cloak URLs and deliver end-users to any end-URL you wish. So, no, you cannot currently match a single cloaked page to a single redirected URL; instead, users are redirected to the same URL. All of the features we have included over the past 2 years are driven by user requests -- so far, we haven't received any requests for this feature.
[signature removed per TOS]
[ Message was edited by: thejenn 09/14/2004 11:30 am ]
|
 |
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-09 04:37
And I really appreciate the hospitality here! It's nice to find a board where anti-cloaking SEOs are willing to discuss it without name-calling, etc. I can't tell you how many other boards I've left because of such stupid barbs thrown at me.
Peter
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-09 04:41
Ok, I think I get you. Your script has a single link, that then leads to all cloaked pages.
I can't just type in a url to a sub page that is flash, and have your script respond, check the IP database and deliver a tailored cloaked page to a crawler.
For example, I create a great flash page that has great content, I find other web sites that agree it's good content, they link to my flash page. I've now got page rank but nothing to index ... your product can't help, because your product is just a link to a cgi program that leads to random generated pages, or with some extra work, I can use a separate "sold separately" add on product that will allow me to write some text, but not really in context to any particular page.
With your product, you can add a link to a web page that leads to ... would "spider trap" be the wrong word?
Bill
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 08:44
Hi "unreviewed": Sorry, I misunderstood. Cloaker is not simply a "spider trap". You can indeed present DIFFERENT HTML content to real users based on the cloaked page they are visiting. Remember, the demo we have on our site is pretty old and doesn't display all the features we have. Here is a screenshot of what I am talking about:
http://www.searchenginecloaker.com/staticexample.jpg
Peter
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 17:31
Ok, I can embed my flash in html, and then write some text.
How do you then link to a specific page? For example would your script need to be installed as the default page to load.
And this is a bit of topic, but I'd really like to hear your opinion on Google. It must have made you very angry when they applied a PR 0 to your web site. Do you think Google is dictating the web?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 20:44
Hi "unreviewed":
You can use Cloaker in many ways. If you wanted to do things expertly, you could use .htaccess to force every visit to ANY URL (or any selected subset) on your site to first run through Cloaker. Cloaker could then look at the URL being accessed and deliver either cloaked pages (for the engines) or "real" content (for human viewers).
I think it is indeed sad that Google, in effect, necessitates how Webmasters must think about how they contruct and design their sites (rather than thinking about end-users first). Despite all of the hype, I feel that their indexing is extremely crude, prompting Webmasters to design Frankenstein-like pages to rank well.
We feel that Cloaker is an outstanding way for responsible Webmasters to throw up their hands and just separately design content for the spiders and separately for real users. Spiders ARE NOT humans, and the bizarre creations that Webmasters increasingly create to appeal primarly to the engines (and secondarily to real users) runs counter to what the Web should be about.
We had an early version of Cloaker running about 2 years ago on our site as a demo. We never intended it to get listed, nor did we really care how our site appeared in Google. I checked the listings one day, and noticed that we had accidentally received something like 200,000+ listings in Google. Yowzers! I promoted this heavily for a while, until we were removed from their index. Because of this, we redesigned Cloaker, and never present a "never-ending" list of URLs like this.
[signature removed, per TOS]
[ Message was edited by: thejenn 09/14/2004 11:31 am ]
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 21:03
If you wanted to do things expertly, you could use .htaccess to force every visit to ANY URL (or any selected subset) on your site to first run through Cloaker.
That's what I'm trying to nail down. Because when I look at your script, it doesn't seem coded to do that. For example, if I use .htaccess and load your program as my default page ... the code as written would be very harmful.
For example, if a crawler visits my domain, your script will give it a page full of text. Great ... however, if it is a human visitor, and your script is set as the default, it then sends a normal page to a human, however, it will seriously slow down my server ... why? Because all calls will need to go through your script if it is set up as default, and that's bad, because if a page contains 10 graphics, your script will make 11 calls to its IP database, one for the html page and one for each graphic, and as the visitor walks through the web site, hundreds of calls to the IP database will result.
Am I incorrect about that, if your script is used and setup as the default homepage?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 21:12
No, this isn't correct.
You would use .htaccess to cause only the root level URL (index.html) or any URL ending in .html (etc.) to run through Cloaker.
Take a look a closer look at .htaccess -- it is a very robust tool to allow precision redirects and rewrites. Our manual gives one generic example on how to use .htaccess to force URLs through Cloaker, but there are a myriad of different things that you can do.
This gets to another point about Cloaker. Our software is both for advanced users and beginning users. The majority of our customers are beginners who have VERY little experience setting up Web sites. It is for this reason that we have opted to make some of the more advanced features as two separate add-ons -- most users don't want these features, or don't really understand them initially. We had huge support headaches when we tried to explain .htaccess to our beginning users who often don't even have a Web host when they buy our software!! Advanced users, on the other hand, are usually fluent in .htaccess, can quickly configure Cloaker to do all sorts of crazy things, and are willing to spend a few dollars more for the advanced add-ons.
[signature removed, per TOS]
[ Message was edited by: thejenn 09/14/2004 11:31 am ]
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 21:17
I should add, that even if the script will ignore graphics, does it have any sort of session handling built in so that it doesn't need to at least make a call to the IP database, for each paged called, in other words, once a visitors IP is established as not a crawler, is it going to call the IP database with every page view?
And for that matter, does it hit the database with every crawler request?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 21:22
No, it is not reliable to rely on "sessions" to track search engines. Many search engines rapidly swap IP addresses for different page visits and do not accept cookies. There is little processing overhead to check each visitor.
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 21:42
Thanks for putting up with me Peter, just a couple more questions about the software, and then if you're interested, I'd like to talk about what matters to many webmasters, and that's ROI, the bottom line.
The IP database ... as you know, you have some tough competition. In fact, for many that may have a reason to cloak, the IP database is the product. Some of your completion is using huge amounts of traffic to identify crawlers, and from what I understand, it isn't something that's easily automated, it's very much a humans job to identify crawler IP addresses. After all, cloaking can be betting the farm. How do you feel your database compares?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 21:48
Our database is updated a few times a week. We have several thousand customer installations, and our users alert us to new spiders that they find.
I do not think that it is the case that the IP database is the product. I think this is sort of like buying into those phonies who say they will submit your site to 20,000 engines. We all know that there are only a few engines that matter, and they reliably indicate who they are through UserAgent.
As such, we use both UserAgent and IP address to catch engines. Almost all are caught on UserAgent (which reduces processing overhead).
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 21:56
Good answer
So it is your experience that cloaker's can depend on the engines to always identify themselves with a UserAgent. That seems pretty sporting of them. 
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 22:09
Yes it is indeed the case that the major engines almost always present UserAgent. The only time that they do not use UserAgent reliably is, for instance, when Google sandbox starts up their own test agents... these sometimes have different names.
We have never seen a spider crawl only to find cloaked pages -- cloaked pages take up a fraction of a fraction of indexed pages. Do they really want to recrawl the whole Web TWICE to find those few cloaked pages? It would be so easily defeatable because of IP detection. And how would they detect it? By sending out two spiders and seeing if they "see" different content? Okay. But how many false positives would result? Basically all dynamically generated pages would be flagged.
Even in the unlikely event a spider finds a non-cloaked page... so what? This isn't how the engines find cloaked pages. The engines appear to be far more concerned with detecting and knocking out visible "spam".
We have had exactly 3 sites removed from the engines, and all of those removals happened about 2 years ago when we had an earlier version of Cloaker that resulted in 100,000+ of listings (and clearly caught their attention).
[Per TOS, signature removed]
[ Message was edited by: thejenn 09/14/2004 11:35 am ]
|
|
unreviewed
Joined: Dec 07, 2000
# Posts: 6776
|
Posted: 2004-Sep-10 22:22
Well that brings to mind, what can the engines do? Google states on thier web site "DON'T USE CLOAKING" ... and they don't do anything to detect it? Nothing?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-10 22:26
As they are very secretive, I don't know what they are doing, but whatever it is, it hasn't seemed to affect our customers. Likely, they are ranking pages more and more on off-the-page factors, again pointing out that cloaking is only a component of an overall Web promotion strategy. They also may be using more "anti-spam" filters, that get rid of ugly doorway pages. This doesn't target cloaking explicitly, but it does weed out those cloakers who try to present poorly designed cloaked pages to the engines (i.e., repeating keywords over and over).
[Per TOS, signature removed]
[ Message was edited by: thejenn 09/14/2004 11:36 am ]
|
|
Ron C
Joined: Jul 23, 1999
# Posts: 1468
|
Posted: 2004-Sep-11 03:57
As such, we use both UserAgent and IP address to catch engines. Almost all are caught on UserAgent (which reduces processing overhead).
So if I spoof the UserAgent (which is obviously much easier than spoofing one of Google's IP addresses), I should be able to detect what your script is serving to the spider?
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-11 05:12
[deleted]
|
|
secloaker
Joined: Sep 08, 2004
# Posts: 19
|
Posted: 2004-Sep-11 05:17
Yes, if you spoof UserAgent or IP address for, say, Google, you will be able to see what search engines see. This isn't a problem in any sense... and if it ever becomes a problem, I would take 5 minutes to put in a toggle allowing users to select between using only UA, IP, or both.
Peter
|
|
Ron C
Joined: Jul 23, 1999
# Posts: 1468
|
Posted: 2004-Sep-11 15:31
I'm honestly surprised it hasn't been a problem, Peter. The greatest danger of any SE spam (a term I hate, btw) usually isn't that Google will detect it, but that your competitors will.
|
|
You are not permitted to post messages in this forum or topic, because of one or more of the following reasons:
- You have not yet logged in, or registered properly as a member
- You are a member, but no longer have posting rights.
- This is a private forum, for which you do not have permissions.
If you are a recent member, it's possible that you simply have not yet confirmed your account. Please
check your email for a message entitled 'JimWorld Forums: Confirm Your Account' and follow the instructions
contained within.
If you cannot find this message, click here to Re-Send it.
|
If you are still experiencing problem, please read the
Login Assistance
Article for some advice on what may be causing your login not to work properly.
|
Switch to Advanced Editor and ...
Create a New Topic
or Reply to this Thread
|
|
Inbox
