What are the search engine risks associated with using automated software. From my experience there are two primary areas of risk: first automated submission of sites and having those sites banned and secondly having a data collection server IP blocked for automated data collection (ranking reports, etc..).

Has anyone ever had issues of getting a site banned in Google (or other search engines) due to running too many reports on it? or has just the IP of the server collecting the data been blocked?

(all of this assuming that the hosting server and the data collection server have different IPs.)

thoughts and experiences on this would be great...

just the IP of the server collecting the data been blocked?

99 percent of the time if you hit a search engine's servers too hard and too often the IP will be blocked.

In extreme cases they've actually banned domains but that most likely won't happen. Besides, if they banned or blocked sites or domains then someone could do it to a competitor and get their site removed. So, the IP is blocked and not the site.

wouldn't that stink to have your domain blocked cause a competitor did too much data collection on it.

so really then, if you can hop between ips to do data collection then there is no risk in using automated tools to collect information about a domain.

is all automated submission bad? we use a number of techniques to get indexed quickly but have always stayed away from automated techniques. Is automation a good solution to get listed with smaller, lesser known search engines and directories?

is all automated submission bad?

No, it's not all bad. There's no need for auto

But, personally I don't really see a need for it; concentrate more on looking at results and ROI and your website stats rather than running automated queries.

I wouldn't submit to the search engines, that's not needed anymore.

[ Message was edited by: bhartzer 08/23/2006 12:17 pm ]

by automated submission I mean submitting a site to a search engine not doing analysis of a site already indexed by the engine.

are you implying that automated analysis should not be done in favor of ROI and web site stats?

for automated submission to be indexed, which are the safest tools?

You do NOT submit to search engines.

Those links you have from other websites to yours are the only thing the engines need to find you.

Regardless of what YOU do, the engines will NOT index you any faster - they do it their way, at thier pace.

If you want to use a tool for directory submissions, that's another matter entirely. Though, with only about 20 or so directories to really bother with (maybe more or less depending on the list you read), doing those directory submissions manually is safe and relatively quick.

SportsGuy is right, some automated submission techniques are now not needed; like auto-submitting to the search engines. It's actually better to let the search engines find you via links to your site.

I guess you could also call the Google Sitemaps program "automated submission", as well, but that's different than using the "add url" feature of some search engines. I personally don't know why they even still have those mechanisms in place.

When it comes to checking rankings, then it's much better to focus on ROI and watching your web stats.

"Those links you have from other websites to yours are the only thing the engines need to find you."

That is what we do now.

I just keep seeing software that submits web sites to search engines (like web position gold) and I was curious if any of it worked or was safe to use.

From a web position gold reseller site

"4. Submit URLs to search engines"

This is what I am talking about when I say automated search engine submission.

what are the best analysis software tools out there?

>> wouldn't that stink to have your domain blocked cause a competitor did too much data collection on it <<

No. They block the people that are doing the data collection.

That's what I figured...