02 February 2009

Why NLP Technology Beats Out SEO Experience in the Long Run

A quick Google search and you quickly learn there are thousands of Search Engine Optimization (SEO) companies out there offering a wide variety of products and services. How can we tell them apart? And how do we know what types of companies will last and what types will take us on as clients today and go out of business tomorrow? To help answer this, I classify them into three categories:
  1. Black-hat SEO Companies
  2. White-hat SEO Companies
  3. Technology Companies Doing SEO
Black-hat SEO Companies
Black-hat SEO (sometimes called spamdexing) is basically using tactics to trick search engines into ranking websites high. This is often considered unethical and is not in the best interest of the search engines or the users. This can be in a variety of ways: creating artificial link farms to trick search engines that a domain is valuable, keyword stuffing certain terms over and over again in content, placing content in an unviewable position or font color so that search engines will index the content, but the user doesn't see it. There are a couple major problems with these flawed approaches:
  • Search Engines (especially Google) are getting very smart about detecting these techniques and actually penalize such websites for attempting them
  • The user experience is compromised because the user is tricked into visiting a website and doesn't find what he is really looking for. So while the website may increase in traffic, sales often don't.
  • The target is constantly moving. Again, because search engines are getting smarter, it is difficult to come up with sustainable techniques.
In short, I wouldn't want black-hat SEO techniques applied to my website, nor would I want to invest in an SEO company using them. Beware of black-hat SEO!

White-hat SEO Companies
White-hat SEO is not trying to trick the search engines or the users. It is simply the art and science (yes, a little of both) of organizing web content and building legitimate web communities to improve search engine rank position (SERP). These are the good guys, the ones the search engines like. These are the guys who use the published SEO guidelines from the search engines, have good netiquette, and not only increase a website's SERP, but also the user experience. As search engine technologies evolve and improve, the user experience is highly correlated to a good SERP.

In short, these are good companies to use as SEO suppliers, but might not be (as I will shortly explain) the best candidates for investors.

Technology Companies Doing SEO
Companies who actually develop technology in areas of natural language processing, web-crawling, text analytics, information retrieval, machine learning, and anything related to understanding text and the web and then apply this expertise to SEO represent the strongest category of SEO companies. These are companies who are deploying white-hat SEO techniques, but in a more intelligent way. The vast majority of white-hat SEO companies are very tactical. They closely monitor the search engines and how the search engine rank positions (SERP) change with every little tweak and adjustment. Some are very good at this. Through experiments and experience, they learn what works. This is not bad, but it is not sustainable. Google and other search engines are constantly working to improve the user experience and are quickly developing new technologies to (as Google puts it) make the world's information universally accessible and useful. This is a moving target! Bets are off as to whether or not SEO companies can keep up with the ever changing technology. Likewise, as the vast number of SEO companies increase, it becomes more and more difficult for them to affect SERPs for all their clients when they compete against other SEO experts.

This is why technology companies have an advantage. Technology companies are working to solve the same problems the search engines are: understanding data and bringing it to users in useful ways. Instead of just using empirical data from search engine experiments, technology companies actually know how to analyze websites and text the same way search engines do. This is a clear competitive advantage over regular SEO companies, who are a dime a dozen. This is how they distinguish themselves from other SEO companies. These are not only great SEO suppliers, but good companies in which to invest.

At Semantic Discovery, we did not start as an SEO company. We started as a technology company in a completely different market. We develop web-crawling technology to capture data from the web. We develop natural language processing and machine learning technology to understand, extract from, and classify webpages. We capture blog data, detect common topics, and detect sentiment related to topics. We use information retrieval technology to index our data and make it searchable and useful. In short, we develop much of the technology that the search engines use to bring users relevant content.

At Semantic Discovery, we soon learned that it made sense for us to start selling SEO products and services because our technology was a perfect fit for SEO! This is a huge advantage for Semantic Discovery in the SEO market!

1 comment:

  1. I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often.
    colin walsh