Search Engine Cloaking
A Marketing Technique You Should Avoid
Originally Published: June, 1999
Despite the fact that this article might cause more controversy than any I have previously written, I feel that this subject needs to be addressed. Before implementing any new marketing strategy, you should ask yourself "Is this the right thing to do?" Although this sounds simple, when dollar signs start dancing around in your head, it becomes easier to look the other way. I know, because I've been tempted on many occasions to choose the wrong path.
The technique known as IP redirection should be considered off-limits to anyone who wants to practice good business ethics. This technique will endanger both the reputation of your business and your ability to advertise on search engines.
Search Engine Cloaking Defined
IP redirection (also known as IP Redirection, user agent cloaking, IP cloaking, search engine cloaking, IP stealthing, page swapping, page redirection, and similar names) is accomplished by running a script on the server which is programmed to look for the IP address of a search engine, and then serve up a page optimized for that engine. However, an entirely different page will be presented to all other visitors.
(
Editor's Note:
These scripts can also be programmed to detect the search engine robot's user agent name.)
At first glance, this may seem like the perfect way to hide your real HTML code from potential competitors. It may also seem like an easier way to create pages designed to rank well since you don't have to be concerned with how the page looks. You can simply list out a mumbo-jumbo of different keywords in the proper quantities and tag locations since the end user will never actually see that page.
What makes this technique wrong? Two fundamental reasons:
1) Cloaking (IP Redirection) Frustrates The Surfer
When someone does a search for a keyword, the pages that are indexed by the search engine are NOT the pages that the search engine returns to the user. This is very frustrating to the Web surfer. It also completely undermines the search engine's attempt to return pages that the user actually searched for. It is one thing to design your pages so that they are appealing to the search engines while displaying content that the user searched for. It is quite another thing to serve up an entirely different page to the engine than what visitors see.
Under ideal circumstances, the two pages are very similar and no great harm is done. However, IP redirection is a technique that is often abused. At a minimum, it causes great confusion and lack of confidence in search engines' accuracy. As an example, we had to create an FAQ in our help file to explain to bewildered searchers why a number of top ranking pages didn't even have the keywords being searched for on the page, or perhaps included the keyword only once on the entire page. IP redirection is a primary reason why this occurs.
Before you say IP redirection does not apply to you, consider other techniques which accomplish almost the same thing. One example are meta refresh tags which causes your doorway page to immediately load a different page when the visitor arrives. Most search engines have caught onto this technique and now simply index the page you're redirecting to, or they ignore the submission entirely. In these cases, you should make sure that the page being redirected to has content that the engine will rank well.
Before I go further, there are some legitimate uses for meta refresh tags or page swapping. You might have an obsolete site where you need to redirect people to the new site or URL. You might also want to serve up one page for Netscape browsers, and another page for Explorer browsers. However, these uses are fundamentally different from displaying one page to the search engine, and another to the visitor.
(
Editor's Note:
See:
Search Engine Safe HTML Page URL Redirect
and
Google PR Improved With Domain Redirection
for search engine friendly ways to handle moved pages & domains.)
2) Cloaking May Get You Banned From The Search Engines
Another compelling reason to avoid IP redirection is using this technique may get you banned from the search engines.
I always try to warn readers about any techniques that can get you banned. Some of these include:
a) Using the same font color as the background to hide keywords.
b) Repeating the same keyword far too many times on the page.
c) Optimizing for keywords that don't apply to your site's content. (A definite ethics violation!)
d) Spamdexing the indexes by submitting too many pages optimized for the same keyword and engine.
I have been contacted by several people in the industry selling or endorsing IP redirection software who were very upset that FirstPlace Software recommended against the practice. These people tried persistently to convince me that:
a) There is nothing wrong with IP redirection.
b) The search engines do not have any problems with the technique.
c) You will never get banned for the practice.
I argued they were mistaken on all three accounts. After debating with them for hours, I agreed to have someone research the issue further just to be certain that what I was recommending to people was the proper thing to do (since they insisted it was not).
I called Fredrick Marckini, President of iProspect.com, Inc. and author of the book "Achieving a TOP 10 Ranking in Internet Search Engines." He agreed to call the major search engines to get their opinion on IP redirection and similar techniques. Fredrick was able to use his contacts to discuss IP redirection with upper management at Lycos, Excite, and HotBot.
A Vice President at Lycos had this to say about IP redirection:
"Our policy is fairly strict; we blacklist the offending site."
"Lycos DOES consider the act of hiding pages via a stealth script to be spamdexing."
"We remove these domains from our database -- from the live search engine database -- and the only time we ever put them back is when we get feedback from the site that, for instance, they fired the Web master or something and it was an honest mistake. Most of the time it comes up in porn [sites]. "
Regarding indexing and blacklisting at Lycos:
1. Lycos tends to blacklist by top-level domain. We could black list one subdirectory, e.g., within Geocities.
2. Lycos employs a "Quality Assurance group" which has the power to blacklist. One reason it takes so long for Lycos to index sites is that they QA the top matches to popular keyword queries -- actual humans review the top matches to remove spam or other irrelevant sites.
3. Lycos claims to now perform a "partial update" every week to two weeks and a complete update every 2 months or so.
I'm no rocket scientist, but I'd say that Lycos is against IP redirection, feels it's wrong, and will ban you if they catch you!
No search engine can logically endorse IP redirection since it completely undermines their ability to rank a page based on its content and the engine's ranking algorithms. This holds true even if the IP spoofer targets only "appropriate" keywords.
People who argue for using IP redirection scripts (usually those who are selling the software, or those making money from it) often will try to turn the tables when they find they're losing the ethical argument. They claim that simply the creation of "doorway pages" is also wrong if judged by the same standards.
However other techniques such as "doorway pages" (pages designed to rank well in search engines, but which ordinary users actually see) use accepted techniques and are ethically sound. The crucial difference is that a doorway page is simply "pleasing" to a search engine, while an IP spoofed page is a different page altogether.
The search engines know you want to rank well and many even include help pages with discussions on the proper use of meta tags. Some include other general tips on optimizing your page. This shows that trying to make your pages rank well is NOT wrong even as defined by the search engines.
Fredrick asked the VP at Lycos about their policy on doorway pages:
"If a doorway page promotes something accurate, then that's great. Let's take a store that sells "lawnmowers" and "kitchen supplies" for instance. What if the home page is all graphics. If you compose a page with some legitimate and accurate copy that describes the Web site that the page is leading people into, then that's perfectly legitimate. You should probably be encouraging people to do this, especially when dealing with a site that has frames, or some another problematic feature that makes properly indexing the page difficult. Lycos will ban pages or sites when they intentionally misrepresent the information contained on the pages, especially when it deals with common queries or irrelevant queries that lead people to an irrelevant site."
HotBot sent us this comment about IP redirection techniques:
"Our basic point of view is that, yes, if you show the indexer one page and the consumer another, even in good faith, you're spamming, and you're annoying us and our users."
The representative at Excite had this to say about IP redirection:
"If the user of the search engine gets a page that's different than what we index, it is not good for our users. It's bad because it harms the user's experience. You could extrapolate from this. In turn, it trickles down to harm the company [Excite]. Excite does not have an official policy dealing with stealth scripting, though. So much of our stuff is automated, we can't go around checking everything."
Thus, at the present time you may not always get banned for using the technique. It is possible that Excite has a procedure that this contact was not aware of. However, whether or not you'll immediately get caught, you must consider if it's ethical. Trying to make an extra buck is not worth losing your integrity.
Search Engine Cloaking Summary
IP redirection could be compared to telling your local Yellow Pages representative that the name of your business is "AAA Jerry's Software Sales" when in reality it is "Jerry's Software Sales." In this case, Jerry lied to the Yellow Page representative by giving them a company name that was not accurate just so he would rank higher in the phone book listings. This is clearly unethical, as is IP redirection.
In closing, you must evaluate whether you're going to run your business based on morals and ethics, or merely upon greed. It is important to set standards based on ethical grounds, not based on the likelihood of being caught or by the actions of others. If you make this type of determination up front, you will avoid slipping into an ethical gray area later on.
Whatever you decide, I'm sure you'll feel more comfortable dealing with a company who although certainly not perfect or immune to error, strives to practice a high level of business principles.
Note: This article was written before Excite became a Meta-Searcher.
Editor's Note:
Some of the search engines in this article have changed but the advise is still good.
You should also watch out for
cloaking by your web host
This article is copyrighted and has been reprinted with permission from FirstPlace Software.
Site Promotion Articles Indexes:
|
