indexing and domain mirroring: a mess

Whoa.

I've been retained as an in house marketing consultant for a corporate Internet firm, and SEO has ended up part of my charge. I'd be reluctant to describe myself as an SEO 'expert', even after fooling with it for the last seven years. But I am somewhat SEO knowledgeable. Clearly though, as often is the case, I'm discovering that the more I learn, the less I know. So I have my hands full.

A case in point. After coming on board the company, I discovered that they have 17 domain names, and 16 of them point to a primary domain that reflects the company corporate name. This has effectively created 16 mirrors pointing to one primary domain where the respective site is hosted on a server. The site consists of nearly 2400 pages, is dynamic, and written in ASPX. All this was done before I entered the scene.

After some study of the SE saturation for each of the 17 domains, and some basic phrase searching, I've also discovered that only one of the 17 domains is being indexed for title, description, and content by Google. It’s definitely and unfortunately not the one reflecting the company name. If I do a Google search on unique quoted phrasing from the site index page, I only get a search result indexing for that one odd domain. All the others, including the primary company name domain, is transparent.

I’ve done a workup of [site:www.domain.com] and [link:www.domainname.com]. Google shows more than 1600 pages of the site has been indexed on the primary company name domain, but does not recognize any content on the index page by that domain. Only one of the alternate 16 domain names. And worse, all of the indexed pages reflect the SAME title, and sadly, all the pages reflect product with unique names already available as variable strings. I had to set the ASPX programmers to fix this and give us in house control over SEO for the respective pages, but the pages have not been reindexed by Google since November. It’s a mess.

Now I can’t begin even a preliminary SEO effort and sort all this out until I figure out how to at least correct the oddity of the indexing of entirely the wrong domain. I really need to get Google to at least see and index content reflecting the primary domain, not necessarily for the other 16, and definitely not the odd one out. I know Google is aware that all the domains exist. If I do a search on the respective 17 domain names themselves as [www.domainname.com], Google shows a single result for each of them, with no title, description, or content reference. All that shows up in the result is the domain itself. The odd domain that does reflect content and such is the exception, where a search result on the domain name returns a standard title, description, and then domain. But it’s not the domain we need to concentrate on.

Some of the other engines have indexed the primary company domain, along with some of the mirrors, although not always well, but not Google. There’s a lot of mixing and matching. I’m thinking this is somehow a result of the mirroring, but I’m not positive. In the past, on more than one occassion, I have seen Google inexplicably drop all indexing for a domain. I've seen positioning go from page one one day to infinity the next. But this one's a bigger mess than I've seen before. I also have no clue where to begin to attempt a correction.

Here's a link to the primary corporate domain:
(url in profile)

Any experts out there with a clue for me? I’d appreciate any insight I can get.


[ Message was edited by: bhartzer 03/07/2005 02:34 pm ]

I know situations like this can become messy very quickly. It happens all the time, though. You're not the first one who has had problems like this.

It sounds like you just need to pick one main domain--then set up 301 Permanent Redirects from all those other domains to the main one.

Since the domain itself is showing up in the index, it sounds like there's a penalty for duplicate/mirrored content. The 301 Permanent Redirects should take care of that.

If your server uses Apache, then this is what you need to add to the .htaccess file:

Options +FollowSymLinks
RewriteEngine on
RewriteCond %{HTTP_HOST} ^domain-one.com [NC]
RewriteRule ^(.*)$ http://www.main-domain.com/$1 [L,R=301]
RewriteCond %{HTTP_HOST} ^www.domain-one.com [NC]
RewriteRule ^(.*)$ http://www.main-domain.com/$1 [L,R=301]
RewriteCond %{HTTP_HOST} ^domain-two.com [NC]
RewriteRule ^(.*)$ http://www.main-domain.com/$1 [L,R=301]
RewriteCond %{HTTP_HOST} ^www.domain-two.com [NC]
RewriteRule ^(.*)$ http://www.main-domain.com/$1 [L,R=301]
....
etc
....
RewriteCond %{HTTP_HOST} ^main-domain.com [NC]
RewriteRule ^(.*)$ http://www.main-domain.com/$1 [L,R=301]

I had a similar problem set, only three domains. Nothing in the magnitude of 16 domains!

The stickler was that some of the extra domains had links ponting to them.

In the end we made the extra domains into 'landing pages' with their own unique and compelling content, to encourage the visitor to click one more time for the main site.

Thanks to everyone who posted to this message. I set the .htaccess to 301 redirects and I'll wait until the next Google crawl to see what happens.

In the meantime, I did a complete search engine saturation check for each of the 17 domains as www.domain.com and domain.com, on Alltheweb, AltaVista, Google, Hotbot, MSN, and Yahoo.

Now the interesting thing is how the respective deep pages were indexed. On some domains I get deep numbers on Alltheweb, AltaVista, etc., and shallow numbers on Google MSN, etc. Then on other domains I get deep numbers on Google and MSN, and maybe one or two on the others.

All these domains pointed to the same server with the exact same link structure and the same content. I don't have the resources to do a 301 redirect on all these pages to keep all this depth of indexing, so most of these will point to the index file on the main domain. I'll loose much of the varied saturation. That's alright by me though, if I can anticipate better content, title, and description indexing in the future on the main domain.

But I'm just completely baffled as to why the depth of the indexing is so random and varied. If I understood this better, I might be able to anticipate and massage the reindexing after the 301 takes effect to get more even saturation depth thoughout the respective engines. I have set into place a methodology to use the dynamic content in my page titles and META descriptions, concatenated with a half dozen basic standardized phrasing for my planned SEO campaigns, so eventually, indexing should occur with some searchable content. (Remember, when I came in, all the dynamic pages, all 2070+ of them, had the SAME title!)

Anyone have any insight into the random and greatly varied depth of page saturation that previously occurred? What should I be thinking about when anticipating reindexing progagation with genuine titles and descriptions?

The randomness occurs because someone linked to your page1.html as domain1.com/page1.html so that is what Google thinks it is called.

When Google arrived at that same content via internal links inside www.domain2.com then it saw that as duplicate content; the direct link into the same page under the other URL probably had a higher PR so it used the other URL.

For your page5.html you had an external inbound link that called it www.domain5.com/page5.html and so that page is indexed under that domain name, and so on. There was nothing there to force the search engine over to www.main-domain.com/page5.html, you had no redirect installed, so it cannot be indexed using the name you really wanted.

The randomness continues as long as your incoming links come and go, and is affected by the ebb and flow of PR around your site.

The cure is to redirect all traffic to the one domain. My .htaccess example above, does just that.



>> I don't have the resources to do a 301 redirect on all these pages to keep all this depth of indexing, so most of these will point to the index file on the main domain. <<

I am not sure why your have complicated this to redirect some parts back to the root. That isn't necessary. The .htacccess example, above, redirects the whole domain to another one, preserving the folder structure and filename in the redirect.

That is a request for domain5.com/folder3/page45.html will be redirected to www.main-domain.com/folder3/page45.html and a request for www.domain3.com/folder7/page89.html will be redirected to www.main-domain.com/folder7/page89.html and so on.