Why NLP Technology Beats Out SEO Experience in the Long Run

A quick Google search and you quickly learn there are thousands of Search Engine Optimization (SEO) companies out there offering a wide variety of products and services. How can we tell them apart? And how do we know what types of companies will last and what types will take us on as clients today and go out of business tomorrow? To help answer this, I classify them into three categories:

1. Black-hat SEO Companies
2. White-hat SEO Companies
3. Technology Companies Doing SEO
   

Black-hat SEO Companies
Black-hat SEO (sometimes called spamdexing) is basically using tactics to trick search engines into ranking websites high. This is often considered unethical and is not in the best interest of the search engines or the users. This can be in a variety of ways: creating artificial link farms to trick search engines that a domain is valuable, keyword stuffing certain terms over and over again in content, placing content in an unviewable position or font color so that search engines will index the content, but the user doesn't see it. There are a couple major problems with these flawed approaches:

• Search Engines (especially Google) are getting very smart about detecting these techniques and actually penalize such websites for attempting them
  
• The user experience is compromised because the user is tricked into visiting a website and doesn't find what he is really looking for. So while the website may increase in traffic, sales often don't.
  
• The target is constantly moving. Again, because search engines are getting smarter, it is difficult to come up with sustainable techniques.

In short, I wouldn't want black-hat SEO techniques applied to my website, nor would I want to invest in an SEO company using them. Beware of black-hat SEO!


White-hat SEO Companies
White-hat SEO is not trying to trick the search engines or the users. It is simply the art and science (yes, a little of both) of organizing web content and building legitimate web communities to improve search engine rank position (SERP). These are the good guys, the ones the search engines like. These are the guys who use the published SEO guidelines from the search engines, have good netiquette, and not only increase a website's SERP, but also the user experience. As search engine technologies evolve and improve, the user experience is highly correlated to a good SERP.

In short, these are good companies to use as SEO suppliers, but might not be (as I will shortly explain) the best candidates for investors.


Technology Companies Doing SEO
Companies who actually develop technology in areas of natural language processing, web-crawling, text analytics, information retrieval, machine learning, and anything related to understanding text and the web and then apply this expertise to SEO represent the strongest category of SEO companies. These are companies who are deploying white-hat SEO techniques, but in a more intelligent way. The vast majority of white-hat SEO companies are very tactical. They closely monitor the search engines and how the search engine rank positions (SERP) change with every little tweak and adjustment. Some are very good at this. Through experiments and experience, they learn what works. This is not bad, but it is not sustainable. Google and other search engines are constantly working to improve the user experience and are quickly developing new technologies to (as Google puts it) make the world's information universally accessible and useful. This is a moving target! Bets are off as to whether or not SEO companies can keep up with the ever changing technology. Likewise, as the vast number of SEO companies increase, it becomes more and more difficult for them to affect SERPs for all their clients when they compete against other SEO experts.

This is why technology companies have an advantage. Technology companies are working to solve the same problems the search engines are: understanding data and bringing it to users in useful ways. Instead of just using empirical data from search engine experiments, technology companies actually know how to analyze websites and text the same way search engines do. This is a clear competitive advantage over regular SEO companies, who are a dime a dozen. This is how they distinguish themselves from other SEO companies. These are not only great SEO suppliers, but good companies in which to invest.

At Semantic Discovery, we did not start as an SEO company. We started as a technology company in a completely different market. We develop web-crawling technology to capture data from the web. We develop natural language processing and machine learning technology to understand, extract from, and classify webpages. We capture blog data, detect common topics, and detect sentiment related to topics. We use information retrieval technology to index our data and make it searchable and useful. In short, we develop much of the technology that the search engines use to bring users relevant content.

At Semantic Discovery, we soon learned that it made sense for us to start selling SEO products and services because our technology was a perfect fit for SEO! This is a huge advantage for Semantic Discovery in the SEO market!

Semantic Keywords: Latent Semantic Indexing and SEO

Latent Semantic Index (LSI), in a nutshell, seeks to index documents (e.g. webpages) based on semantically related concepts and keywords, not just syntactically related keywords.

Let me illustrate with a couple simple examples. We all have seen how Google recognizes that when you search for "car", you might also want webpages that contain "cars". This is a syntactical similarity - plural / singular, different tenses of a verb, etc. Search engines have long used syntactical variations on searches. Makes sense, seems pretty obvious. Latent Semantic Indexing, however, takes this to the next level. LSI basically learns from the web what keywords are most commonly related to "cars", for example. Then these keywords, which may or may not be synonyms for cars, are used in ranking webpages. If, for example, the top related keywords to "car" are "used cars", "new cars", "car rentals", and "car reviews", then perhaps when I search for "car", webpages with these additional keywords might rank higher than a page that mentions "car" but does not contain any of the commonly related keywords.

Why LSI and what does it mean for Search Engine Optimization? Both questions are answered the same way. Search engines seek to optimize the user experience by providing the most relevant, authoritative content for any given search. Search Engine Optimization (SEO) is our effort to organize our information in a way that search engines can easily understand so that it ranks high for the keywords we are targeting. Both work together to ultimately bring people to our websites who are looking for what we have to offer. How to do this is both the art and science (and yes, a little of both) of SEO.

So why LSI? Latent Semantic Indexing gives the search engines more information about the content of a webpage and helps rank the pages better. I might create a webpage about cars, but only use the keyword "car" a couple times, but when I am describing either the car parts, the economy around the car industry, or issues related to car rentals, I am using language specific to those subjects. Basically, LSI uses natural language processing techniques to learn this language. Now, using LSI, a search engine can take webpages that contains the keyword "car" and differentiate which ones are more relevant based on what other keywords are used.

Example:

I drove to school in my car, took my test, then came home early.

I can't decide if I want to buy a new or used car. Since the accident, I have been driving a rental.

Both of the above sentences mention the word "car" one time, but the former is not about a car at all, whereas the latter is entirely about a car. In fact, typical Information Retrieval techniques would rank the first sentence higher because it is shorter (meaning the word "car" represents a larger percentage of the total content). Using LSI techniques, a search engine might learn "new", "used", "rental", and "accident" are related to "car" and therefore rank the webpage containing the second sentence higher than that of the first when someone searches for "car".

LSI enables search engines to better rank relevant webpages to searches. There is plenty of evidence to suggest that LSI techniques are being used more and more by Google and other top search engines. This does have an effect on SEO techniques.

At Semantic Discovery, we combine our web crawling and natural language processing expertise into building SEO tools that really make a difference. We have developed a keyword suggestion crawler tool that employs Latent Semantic Indexing techniques. Our keyword suggestion tool doesn't just provide syntactic variations of keywords. There are plenty of free keyword tools out there that do that. Instead, what we focus on is bringing back the top related concepts to the keywords you are targeting. If the search engines are using LSI techniques to rank pages, then why not use LSI techniques to learn what related keywords are best to use?

Try the SD Keyword Suggestion Crawler for free!

Introducing Semantic Discovery

Welcome to Semantic Discovery's new blog! Our Semantic Discovery team has been working hard and is very excited to share our great new SEO products with the world. We have combined our expertise in web crawling and natural language processing to bring you our first two of many new SEO tools.

The first SEO tool we offer is our Site Visibility search engine ranking report. This tool allows you track the Google search engine rank of up to 50 keyword search terms for as many websites as you choose. The Site Visibility search engine ranking report can also track the Google ranking of up to five your competitors, and will auto discover an additional five competitors of your keyword search terms. This great tool is available immediately for subscription, or learn more by visiting our website.

Our second SEO tool is our Keyword Crawler keyword research tool. Keyword Crawler is actually two great keyword research tools bundle together; Keyword Site Crawler and Keyword Suggestion Crawler.

• Keyword Site Crawler - Analyze your competition by utilizing the Keyword Site Crawler to crawl a group of websites, perform keyword analysis on the results, and provide a list of the keyword phrases being used on their most relevant pages.
• Keyword Suggestion Crawler - Reach a broader audience by utilizing the Key Suggestion Crawler to find the current top semantically related keyword search phrases on the web from a list of your own keyword search phrases.
  

Both Keyword Crawler tools are available immediately for subscription, or learn more by visiting our website.

Bundle and save money by subscribing to both the Site Visibility search engine ranking report and the Keyword Crawler keyword research tools.

Please visit our blog often for news about Semantic Discovery, SEO tips, details about our existing and future SEO products, and thoughts on our core expertise in natural language processing, web crawling, and corporate information services.