cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/03/2008 01:40 am
Sorry if this is the wrong forum but I'm not sure where this should go.
I get A LOT of hits on our site with the following referer string:
search.live.com/results.aspx?q=night&mrt=en-us&FORM=LIVSOP
Its messing up my visitor statistics because it now looks like I have thousands of hits from search.live.com with the search phrase "night" (my website has nothing to do with that and doesn't come up if you run the search manually)
Any idea what it could be? I do get hits from search.live.com with other keywords...
|
 |
Dinkar
Staff
Joined: Aug 12, 2001
# Posts: 4356
|
Posted: 01/03/2008 02:26 am
Are all hits from same IP? In that case just ban the IP.
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/03/2008 02:50 am
Found the problem in the link below:
sebastians-pamphlets.com/msn-admits-clueless-and-ineffective-spamming/
Absolutely unbelievable that Microsoft are 'pretending' to be real visitors when crawling sites.
|
|
g1smd
Staff
Joined: Jul 28, 2002
# Posts: 10288
|
Posted: 01/03/2008 05:10 am
I missed the connection with that question, and didn't make the connection with Microsoft, even though I was aware of Microsoft being "up to something".
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/03/2008 05:46 am
Well put simply it seems that Microsoft's bots don't identify themselves as bots, instead they 'pretend' to be real visitors coming from a search run on live.com, and mess up everyone's visitor statistics.
|
|
mo007
Joined: Dec 13, 2005
# Posts: 52
|
Posted: 01/03/2008 07:14 am
Could MS really do something like this? And why would they hide?
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/03/2008 08:13 am
My visitor stats logs don't lie... and all from MS IP Addresses
|
|
beth_lk
Staff
Joined: Jun 23, 2004
# Posts: 1139
|
Posted: 01/03/2008 09:00 pm
WoW!! In case anyone else runs into this - does anyone out there know how to correct it?
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/03/2008 11:34 pm
Well I've had to hack my visitor stats to ignore hits from that bot, and I'm going to have to keep an eye on all of our client's websites in case the same thing happens to them.
You could just ban the IP address, but I haven't seen fit to ban anyone from our site yet.
|
|
beth_lk
Staff
Joined: Jun 23, 2004
# Posts: 1139
|
Posted: 01/03/2008 11:40 pm
I am confused....
I thought your counts which appear to be visitors were actual bolts from MSN search engine spider or such.
If I am correct then banning anyone would not help, and would be wrong to do. As you can't ban MSN, can you?
I think maybe I misunderstand what is happening - sorry
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/04/2008 12:32 am
Well precisely, banning Microsoft is hardly a good idea now is it.
However, this isn't the main MSN bot (at least I don't think it is) its some kind of 'quality control' program (according to Microsoft's statement anyway)
However the weird thing is its been going on for months, I first noticed these hits in my logs back in August 2007! I can't believe they're still doing it.
Of course I understand they want to pretend to be 'real' visitors to see if any websites employ underhand tactics to optimise for search engines. But in the meantime its causing a disruption to every other legitimate website in the world.
I wouldn't even have minded if they'd have let me know in advance, they can get our email address from our website easily enough. But they've been doing this for 6 months or more now and we've been thinking our sites have gotten more hits than they really have. When you have a business which relies on these hits it stops being funny.
Check out these logs, these where from 1 single day, on 18/10/2007 we got 67 hits from search.live.com spread throughout the day, all of them from the MS search bot, and all of them with the same Browser string. Note these are UNIQUE hits, the same bot was hammering the website all day long
Browser: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)
Date DomainName SearchTerm
18/10/2007 03:03:28 search.live.com homes
18/10/2007 03:31:59 search.live.com night
18/10/2007 03:32:07 search.live.com night
18/10/2007 03:32:21 search.live.com night
18/10/2007 03:32:22 search.live.com night
18/10/2007 04:04:16 search.live.com night
18/10/2007 04:04:29 search.live.com night
18/10/2007 04:08:46 search.live.com night
18/10/2007 04:09:07 search.live.com night
18/10/2007 04:10:28 search.live.com night
18/10/2007 04:10:38 search.live.com night
18/10/2007 04:10:50 search.live.com night
18/10/2007 04:11:29 search.live.com night
18/10/2007 04:32:13 search.live.com night
18/10/2007 04:45:23 search.live.com night
18/10/2007 04:45:23 search.live.com night
18/10/2007 04:46:58 search.live.com night
18/10/2007 05:01:46 search.live.com night
18/10/2007 06:10:40 search.live.com night
18/10/2007 06:11:20 search.live.com night
18/10/2007 06:35:25 search.live.com night
18/10/2007 06:36:19 search.live.com night
18/10/2007 06:36:24 search.live.com night
18/10/2007 06:40:19 search.live.com night
18/10/2007 08:12:00 search.live.com night
18/10/2007 08:12:47 search.live.com night
18/10/2007 08:13:31 search.live.com night
18/10/2007 10:02:06 search.live.com night
18/10/2007 10:26:22 search.live.com disney
18/10/2007 10:27:22 search.live.com night
18/10/2007 10:29:35 search.live.com india
18/10/2007 10:36:29 search.live.com night
18/10/2007 10:39:40 search.live.com night
18/10/2007 10:42:58 search.live.com night
18/10/2007 10:49:59 search.live.com night
18/10/2007 11:42:58 search.live.com night
18/10/2007 11:43:09 search.live.com night
18/10/2007 12:26:20 search.live.com night
18/10/2007 13:47:13 search.live.com night
18/10/2007 16:04:28 search.live.com night
18/10/2007 16:04:44 search.live.com night
18/10/2007 16:16:03 search.live.com night
18/10/2007 16:44:03 search.live.com night
18/10/2007 18:47:34 search.live.com night
18/10/2007 18:47:35 search.live.com night
18/10/2007 19:20:37 search.live.com night
18/10/2007 20:41:23 search.live.com night
18/10/2007 20:51:25 search.live.com night
18/10/2007 20:54:39 search.live.com night
18/10/2007 21:39:29 search.live.com night
18/10/2007 21:52:31 search.live.com night
18/10/2007 21:52:51 search.live.com night
18/10/2007 21:53:39 search.live.com night
18/10/2007 22:03:44 search.live.com night
18/10/2007 22:07:15 search.live.com night
18/10/2007 22:08:53 search.live.com night
18/10/2007 22:09:41 search.live.com homes
18/10/2007 22:35:21 search.live.com night
18/10/2007 22:39:59 search.live.com homes
18/10/2007 23:00:20 search.live.com night
18/10/2007 23:22:21 search.live.com disney
18/10/2007 23:22:39 search.live.com night
18/10/2007 23:24:00 search.live.com night
18/10/2007 23:30:25 search.live.com night
18/10/2007 23:32:14 search.live.com disney
18/10/2007 23:32:23 search.live.com night
18/10/2007 23:38:09 search.live.com night
[ Message was edited by: cusimar9 01/04/2008 12:44 am ]
|
|
Prowler
Staff
Joined: Aug 14, 2000
# Posts: 1752
|
Posted: 01/14/2008 12:20 am
I really don't see why anyone would be upset at 67 requests spread over a day. I have seen some search engine robots pounding a site with thousands of rapid fire requests - which surely would worry the server admin and the owners.
Unless the requested files consume a large amount of bandwidth, it doesn't warrant any action. Many times some browsers would lop off long referrer strings and incidentally MSN/Live referrers seem to suffer the most.
|
|
cusimar9
Joined: Jan 03, 2008
# Posts: 7
|
Posted: 01/14/2008 12:27 am
I don't have a problem with 67 hits spread throughout the day, its the fact that the bot PRETENDS TO BE A VISITOR and doesn't identify itself as a bot. Now I know I can ignore its visits, but until I worked it out I thought my website was getting thousands more hits than it really is.
This website is a business who's budget depends on these visitors and we've been close to pulling the plug on it. Perhaps it wouldn't matter on most websites but it did matter at that particular time on ours.
|
|
