Anyone know what this referer is?

Sorry if this is the wrong forum but I'm not sure where this should go.

I get A LOT of hits on our site with the following referer string:

search.live.com/results.aspx?q=night&mrt=en-us&FORM=LIVSOP

Its messing up my visitor statistics because it now looks like I have thousands of hits from search.live.com with the search phrase "night" (my website has nothing to do with that and doesn't come up if you run the search manually)

Any idea what it could be? I do get hits from search.live.com with other keywords...

Are all hits from same IP? In that case just ban the IP.

Found the problem in the link below:

sebastians-pamphlets.com/msn-admits-clueless-and-ineffective-spamming/

Absolutely unbelievable that Microsoft are 'pretending' to be real visitors when crawling sites.

I missed the connection with that question, and didn't make the connection with Microsoft, even though I was aware of Microsoft being "up to something".

Well put simply it seems that Microsoft's bots don't identify themselves as bots, instead they 'pretend' to be real visitors coming from a search run on live.com, and mess up everyone's visitor statistics.

Could MS really do something like this? And why would they hide?

My visitor stats logs don't lie... and all from MS IP Addresses

WoW!! In case anyone else runs into this - does anyone out there know how to correct it?

Well I've had to hack my visitor stats to ignore hits from that bot, and I'm going to have to keep an eye on all of our client's websites in case the same thing happens to them.

You could just ban the IP address, but I haven't seen fit to ban anyone from our site yet.

I am confused....

I thought your counts which appear to be visitors were actual bolts from MSN search engine spider or such.

If I am correct then banning anyone would not help, and would be wrong to do. As you can't ban MSN, can you?

I think maybe I misunderstand what is happening - sorry

Well precisely, banning Microsoft is hardly a good idea now is it.

However, this isn't the main MSN bot (at least I don't think it is) its some kind of 'quality control' program (according to Microsoft's statement anyway)

However the weird thing is its been going on for months, I first noticed these hits in my logs back in August 2007! I can't believe they're still doing it.

Of course I understand they want to pretend to be 'real' visitors to see if any websites employ underhand tactics to optimise for search engines. But in the meantime its causing a disruption to every other legitimate website in the world.

I wouldn't even have minded if they'd have let me know in advance, they can get our email address from our website easily enough. But they've been doing this for 6 months or more now and we've been thinking our sites have gotten more hits than they really have. When you have a business which relies on these hits it stops being funny.

Check out these logs, these where from 1 single day, on 18/10/2007 we got 67 hits from search.live.com spread throughout the day, all of them from the MS search bot, and all of them with the same Browser string. Note these are UNIQUE hits, the same bot was hammering the website all day long

Browser: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; .NET CLR 1.1.4322)

Date DomainName SearchTerm

18/10/2007 03:03:28 search.live.com homes
18/10/2007 03:31:59 search.live.com night
18/10/2007 03:32:07 search.live.com night
18/10/2007 03:32:21 search.live.com night
18/10/2007 03:32:22 search.live.com night
18/10/2007 04:04:16 search.live.com night
18/10/2007 04:04:29 search.live.com night
18/10/2007 04:08:46 search.live.com night
18/10/2007 04:09:07 search.live.com night
18/10/2007 04:10:28 search.live.com night
18/10/2007 04:10:38 search.live.com night
18/10/2007 04:10:50 search.live.com night
18/10/2007 04:11:29 search.live.com night
18/10/2007 04:32:13 search.live.com night
18/10/2007 04:45:23 search.live.com night
18/10/2007 04:45:23 search.live.com night
18/10/2007 04:46:58 search.live.com night
18/10/2007 05:01:46 search.live.com night
18/10/2007 06:10:40 search.live.com night
18/10/2007 06:11:20 search.live.com night
18/10/2007 06:35:25 search.live.com night
18/10/2007 06:36:19 search.live.com night
18/10/2007 06:36:24 search.live.com night
18/10/2007 06:40:19 search.live.com night
18/10/2007 08:12:00 search.live.com night
18/10/2007 08:12:47 search.live.com night
18/10/2007 08:13:31 search.live.com night
18/10/2007 10:02:06 search.live.com night
18/10/2007 10:26:22 search.live.com disney
18/10/2007 10:27:22 search.live.com night
18/10/2007 10:29:35 search.live.com india
18/10/2007 10:36:29 search.live.com night
18/10/2007 10:39:40 search.live.com night
18/10/2007 10:42:58 search.live.com night
18/10/2007 10:49:59 search.live.com night
18/10/2007 11:42:58 search.live.com night
18/10/2007 11:43:09 search.live.com night
18/10/2007 12:26:20 search.live.com night
18/10/2007 13:47:13 search.live.com night
18/10/2007 16:04:28 search.live.com night
18/10/2007 16:04:44 search.live.com night
18/10/2007 16:16:03 search.live.com night
18/10/2007 16:44:03 search.live.com night
18/10/2007 18:47:34 search.live.com night
18/10/2007 18:47:35 search.live.com night
18/10/2007 19:20:37 search.live.com night
18/10/2007 20:41:23 search.live.com night
18/10/2007 20:51:25 search.live.com night
18/10/2007 20:54:39 search.live.com night
18/10/2007 21:39:29 search.live.com night
18/10/2007 21:52:31 search.live.com night
18/10/2007 21:52:51 search.live.com night
18/10/2007 21:53:39 search.live.com night
18/10/2007 22:03:44 search.live.com night
18/10/2007 22:07:15 search.live.com night
18/10/2007 22:08:53 search.live.com night
18/10/2007 22:09:41 search.live.com homes
18/10/2007 22:35:21 search.live.com night
18/10/2007 22:39:59 search.live.com homes
18/10/2007 23:00:20 search.live.com night
18/10/2007 23:22:21 search.live.com disney
18/10/2007 23:22:39 search.live.com night
18/10/2007 23:24:00 search.live.com night
18/10/2007 23:30:25 search.live.com night
18/10/2007 23:32:14 search.live.com disney
18/10/2007 23:32:23 search.live.com night
18/10/2007 23:38:09 search.live.com night

[ Message was edited by: cusimar9 01/04/2008 12:44 am ]

I really don't see why anyone would be upset at 67 requests spread over a day. I have seen some search engine robots pounding a site with thousands of rapid fire requests - which surely would worry the server admin and the owners.

Unless the requested files consume a large amount of bandwidth, it doesn't warrant any action. Many times some browsers would lop off long referrer strings and incidentally MSN/Live referrers seem to suffer the most.

I don't have a problem with 67 hits spread throughout the day, its the fact that the bot PRETENDS TO BE A VISITOR and doesn't identify itself as a bot. Now I know I can ignore its visits, but until I worked it out I thought my website was getting thousands more hits than it really is.

This website is a business who's budget depends on these visitors and we've been close to pulling the plug on it. Perhaps it wouldn't matter on most websites but it did matter at that particular time on ours.