What's spam? Search columnist Kevin Ryan wasn't quite correct in saying there are no standards in his recent article. Indeed, the two major search providers each publish standards about what they find unacceptable. Here they are: Google's and Yahoo's.
Ryan's summary of things to avoid, like other ones that have appeared recently from Shari Thurow (here and here) Dave Wallace and Jill Whalen, is a good one.
Clearly, knowing what's widely considered spam isn't hard. So why aren't people "playing by the rules?" Here are just a few reasons:
* The rules for each search engine aren't necessarily the same, though they do seem closer today than in the past.
* Some people have a "the cheaters are winning" mentality. If they see spam slip through, they feel like they should do the same (regardless of the fact that the "cheat" technique might not actually be what's helping a page rank well).
* Some people simply don't agree with the search engine rules. For example, they might justify the use of "hidden text" via cascading style sheets to make up for the fact that their home page has no text. To them, search engines are simply stupid for not accepting this.
* Some people simply don't care. As far as they're concerned, as long as a person gets to something relevant to their search topic, who cares how it came up?
How About Standards?
Ryan's commentary sees one solution to spam as a lobbying for standards. This idea has been floating around for ages and has gone nowhere.
SEM pioneer Paul Bruemmer pushed for search engine optimization certification back in 1998. But as I wrote then, just having a "rule book" doesn't mean and end to spam.
We also had a push in 2001 for search engine marketing standards, which also has gone nowhere in terms of reducing spam in search engines.
How About Better Search Engine Disclosure?
Want a real solution to the spam problem? Then let's have the search engines agree to publish lists of firms and companies that they have banned. That would help the consumer seeking an SEM firm to understand which firms to avoid. Or, if they do use a banned firm, at least they've been warned about the consequences if they still want to go with a "rule" breaker.
It's something I've suggested before, and the search engines themselves have discussed the idea at various times in the past. It's never gone forward, because the search engines seem fearful of legal actions, should they out-and-out call a firm a spammer.
Given this, it's with some sympathy that I've defended the still new SEMPO search engine marketing industry group, when it has come under fire for not trying to ensure its members adhere to search engine spam guidelines.
SEMPO recently posted a FAQ explaining why it has declined to do this. I feel even more forcefully. If the search engines aren't brave enough to enforce their own laws, why should the onus be on a third party group that doesn't even create these rules?
They Do Enforce!
Of course, search engines do police for spam. If they catch it, a page might be penalized for banned entirely. But that's not the same or as effective as providing an offenders list, for a variety of reasons.
* An offenders list ensures that people who aren't ranking well or aren't listed for perfectly innocent reasons can discover that their problems are NOT due to a spam penalty. Far too often, people assume that they've "accidentally" spammed when they haven't. That leads them to perhaps make changes they needn't do.
* Sometimes spam is simply allowed to continue on. Google is especially famous for this, preferring to seek "algorithmic" solutions to removing spam than perhaps to react immediately to spam reports and yank material. Why? Google says it wants to detect overall patterns and come up with more comprehensive solutions. Unfortunately, the waiting period for this fuels the "anything goes" fears that some search marketers have when they see spam escaping prosecution.
* Disclosure helps searchers, not just companies that want to be listed. We've seen press outcry over filtering of adult content or filtering content in response to national laws, with the idea that perhaps Google and other search engines should disclose what they remove. But as I've written several times before, no search engine discloses what they've removed for spam reasons. That's something a searcher might want to know.
Disclosure...
To further expand on my last point, Google currently provides you with two ways to discover if they've removed material because of the Digital Millennium Copyright Act, a US law that can cause search engines to bar listings. A DMCA case involving the Church of Scientology in 2002 is probably the most famous example of this.
Google provides DMCA takedown requests that they receive to the Chilling Effects Clearinghouse. So, do a search there for Google, and you can review the material Google's been asked to remove.
I don't believe that Yahoo does the same thing, provide copies of DMCA takedown requests. However, you can get some sense by searching for Yahoo at Chilling Effects. Ironically, this brings back requests to Google that were probably bulk directed to Yahoo and other search engines.
Google will also provide "inline" notification if it has removed or suppressed material that might otherwise have shown up in the results you were reviewing. For example, try a search for kazaa. At the bottom of the page, you'll see this notice:
In response to a complaint we received under the Digital Millennium Copyright Act,
we have removed 5 result(s) from this page. If you wish, you may
read the DMCA complaint for these removed results.
This is excellent disclosure (and not something I believe Yahoo or other search engines do). Google's not only telling you that they removed results, but you can also clickthrough to read the actual complaint that they reacted to.
...And Lack Of Disclosure
Now return to the spam issue. As a searcher, were you informed that material was removed from your search request? Nope. And might there be an economic incentive for search engines to ban sites? Absolutely. Removing sites may cause those sites to resort to taking out ads, exactly the accusation that Google came under (and strongly denied) at the end of last year, when many sites lost rankings.
Let's not single out Google. Long-time search engine marketer Greg Boser nearly brought an audience of search engine marketers to its feet in applause when he criticized search engines themselves as needing better standards during a session Search Engine Strategies in 2002.
One of the issues Boser complained about was how sites would get pulled, only to then hear from various search engines about how they could get back in in through advertising or paid inclusion.
Real-Life Disclosure Example
Want a real-life example of the need for disclosure? Earlier this month, NetIQ, the maker of WebTrends, purchased rank checking tool WebPosition. It was a pretty big deal. Perhaps someone may have gone to Google and done a search for webposition to find out more about the software.
Good luck finding the official WebPosition site. It's not in the top results, unusual given that Google built its reputation largely on providing good navigation to official sites. Why not? Because WebPosition has no pages listed in Google at all. For ages, I've not seen Google list pages from WebPosition. It's probably banned, but of course Google doesn't confirm these things. And as a searcher, it's not something that was disclosed to you.
The Google-WebPosition problem? Google doesn't like the burden the popular rank checking software places on its system, so explicitly warns people not to use it. But ironically, you'll notice that Google has no problems taking ads for the software from WebPosition's many resellers.
Let me stress -- similar things are happening at other search engines as well. The need for better spam disclosure is universal, not just a Google-specific problem.
Protect Yourself
Better disclosure would be helpful for so many reasons. It would help confused web site owners. It would help guide those seeking the many good, search engine marketing firms that diligently try to avoid trouble and play by the search engine rules. I'd love to see it happen.
Until it does -- or more likely, given that it may never come -- here are some additional suggestions to guide you.
* When it comes to spam, the search engines are the lawmakers. You operate within their borders and are subject to whatever rules they may or may not publish. Break the rules, and you can get tossed out of the country. So learn the rules, as much as you can -- assuming you want to avoid trouble.
* Outsourcing? Knowing the rules may not help, if the SEM company simply makes up its own euphemisms to cover up things search engines don't like. So get references. You might also try contacting the search engines themselves for advice about the company, but don't hold your breath waiting for a response. Still do it. Should you get banned, at least you have some evidence that you tried to do due diligence with the search engine themselves.
Looking for more on the topic of spam? Search Engine Watch members have access to the Search Engine Spamming page, which has a summary of things commonly considered bad to do, as well as a compilation of articles on the topic. The Search Engine Optimization Articles and Search Engine Marketing Articles pages also have a compilation of articles that touch on spam and past guidelines and standards attempts.
Another new spam resource is a scholarly paper that's just appeared out of Stanford, Web Spam Taxonomy. It's a nice summary, though not entirely accurate. Google still stands by the fact that it does not give out "URL spam" penalties for URLs that contain to many keywords in them (though I personally wouldn't do this, despite such assurances). And the example giving a meta keyword tag spam example in the report actually doesn't look like spam at all. But only a search engine could tell you for certain -- and none of them do.
For fun, you might also look at the new Black Hat SEO site. This is a humorous directory to unsavory search engine marketing pitches that have been received by Aaron Wall.
Finally, the Outsourcing Search Engine Marketing page is a compilation of articles covering advice on seeking out SEM firms to work with, such as the recent SEO outsouring one we published on this topic in SearchDay last month.
http://searchenginewatch.com/showPage.html?page=3344581
