1997 – 1999 were the early years of search engine era. SEO mainly consisted of submitting your sites to the search engines. These "voices from the past" are still heard today when this or that SEO company and / or automatic submitting software will claim to do search engine promotion for you by submitting your site to hundreds and thousands engines and directories. The search engines' indexing programs, called "robots" or "spiders", looked through all of the HTML code of a page and used some page ranking algorithms that they kept in secret.

Those days were spammer's Heaven: it was rather easy to get your site ranked high. You could just use your keywords lots of times on the page, in the META tags, HTML comments etc. and hide it from the human visitors by making the text tiny or completely invisible with the help of HTML tricks. The search engines didn't have any sophisticated technique to recognize this kind of spam, and such sites usually got high rankings very easily. Today, you still can find some samples of this primitive optimization (however you will have to give it a hard try, because nowadays most of such Web projects have been banned by the search engines for excessive keyword usage).

The only exception was Yahoo which has always been indexed by humans who could, in most cases identify and ban spamming pages.

Gradually, search engines started recognizing spam and applying corresponding penalties to Web pages using spam methods. However, search engine optimizers were always one step ahead of the search engines in finding new ways of cheating the indexing algorithms. Since each search engine is committed to delivering only relevant results to its visitors, the engines needed to take control away from the spammers and auto-submitters. Many began to try different ways of indexing.

By mid-1999, search portals began using the logic of an ordinary Web surfer to improve the quality of search results. Before this, search engines ranked pages primarily by keyword distribution, which was easily manipulated by spammers. Another way was needed for search engines to estimate a site's importance. Thus, algorithms were developed that utilized click tracking. Sites receiving more clicks from particular keyword search results were considered more important, and ranked higher for subsequent searches based on those keywords. DirectHit (since acquired by AskJeeves / Teoma) introduced a technology that watched which sites were chosen by Web surfers. If the surfers often choose a certain Web site for a given keyword, this site will rise in position for that keyword.

Another way to make indexing algorithms more bulletproof is to rank pages based on how many other pages are linked to it. This "reference" principle has come from regular libraries and archives and now plays the leading role in governing the traffic flow in the Internet. It has been named "link popularity" and remains a huge factor in ranking.

Both of the above parameters are known as "off-the-page" factors because they are factors that are not directly based on the content on your Web page. The claim of "off-the-page" factors is being liberated from the spammers' influence.

The Search Engine Optimization industry has found a temporarily appropriate response by creating so-called "link farms". The idea is the following: if the search engines consider how many inbound links a Web site has got, let's create a special link page outside our site which is of low interest to the visitors but which can be fed to the spidering robot. We can place many links to our site on this page and on lots of similar pages, and these links will have their say when the robot comes to index.

The link to a link farmer's Web site was placed on a great number of similar link pages of other participants, and in exchange this farmer had to place their links on his own link page . The official aim of the link farms was to create a community united by total cross-linking, however the hidden goal was achieving high search engine rankings.

While link farms were enjoying great popularity even until recently, and loads of special software for link exchange between the link farmers has been written, the search engine spiders became aware of this long ago. The contemporary search engines don't care a straw about these kinds of link pages when ranking a site.

Yahoo was the first and is still one of the most popular search engines. Primarily a human-edited directory, Yahoo faced the problem of finding human resources to deal with an exponentially increasing number of pages queued for indexation. While human-edited directories provide quality results and are almost spam-insensitive, the limited number of editors makes it troublesome to achieve the same perfection in quantity.

Community-edited directories first appeared in 1999. The concept allows for thousands of editors, organized in a system of self-governance, to constantly improve and expand the directory with the new resources. The first to come was the Netscape Open Directory; the Go.com directory (now not functioning) was another early leader. Zeal.com, which feeds results into LookSmart.com and MSN search, is a newer addition to this category.

The Netscape Open Directory, besides being community-edited, also was a kind of open source directory. Any developer wanting to create or improve their own search portal had access to its content. This resulted in the fact that by the year 2000, listings from Netscape Open Directory started showing up on almost every major search engine.

The community-edited directories have developed a successful junction of quantity and quality, becoming a weighty part of the search world. Human-edited directories in general tend to play the role of "searcher advocate", since they produce very relevant results for any given search. With the rise in importance of directory listings in 2000, search engine marketers began to concentrate on optimizing their sites for focused, targeted, and quality content.

In 1999, AltaVista has made an attempt to introduce paid listings, i.e. to take fees for inclusion into the search index. This attempt was totally uncompetitive and was universaly denounced. Alta Vista soon quit that project. Nevertheless, by the end of 2000 all major search engines offered some kind of paid listing option. One of the brightest current examples is Overture.com (previously Goto.com), which began as an independent search player and is now owned by Yahoo. It is still one of the major paid listing / advertising resources, offering ranking for keywords based on an auction system: the higher the bid for a certain keyword, the higher your rank for this keyword. The bid amount is charged every time a user clicks on your site listing, a payment model known as Pay-Per-Click (PPC).

In 2001 most directories and search engines introduced various payment models for paid listings: fees for indexing by the search engine spiders, fees for listings in human-edited directories, ad placement opportunities etc. Yahoo established a one-time submission fee, which was later changed to a yearly submission fee. LookSmart switched from a one-time submission fee to a PPC model. Spider-based engines such as FAST, AltaVista, and Inktomi introduced paid inclusion, i.e. a recurring fee that ensures a site will be listed and regularly re-spidered.

Ask someone today to name a search engine, and the answer will be most likely be Google. Google started its rise toward dominance of the search engines in 2000 By 2002 this was firmly established with around 70% of searches done on the Internet. While other search engines were focusing on transforming to universal portals, Google kept a simple and –its most distinctive feature – fast interface that solely targeted delivering relevant search results.

Google also developed advanced features such as indexing and searching PDF (portable document format) and SWF (shockwave flash) files. Additionally, Google's sophisticated techniques used "off-the-page" factors making it extremely spam-resistant. Google began its dominance in 2000 when Yahoo switched from Inktomi to Google as its secondary search result provider. Now, Yahoo uses the combination of Overture's own search software and index repository, thus being fully independent of Google. However, this hasn't had the slightest impact on the dominance of Google, which continues to reign supreme in the search engine world.

By 2001, the results of all major engines were produced from a number of mixed / hybrid sources. Yahoo search results combined Yahoo-directory listings, Overture (PPC) results, and Google results. MSN provided results from Overture (PPC), LookSmart , and Inktomi.

In 2002 and 2003 major reshuffling happened among search engines. In this period, Google purchased Blogger.com, AltaVista and AllTheWeb became a part of Overture, and Yahoo bought Inktomi. There were also many shifts caused by emerging search engine partnerships. Further in this course we will give you a complete chart outlining the relationships between the contemporary search engines.

Here's a table reflecting the major events in the search engine world in 2002-2004:

 Search Engine Marketing Today

If you still think that search engine marketing can be done by acquiring (and using) auto-submission software, drop this idea now. Search engine marketing requires an integrated approach to improve site content, quality and popularity. For a Web site to reach its top potential, it must incorporate target audience analysis, competitive analysis, cost per click optimization, and - last but not least - copywriting and copyediting. And, because things keep changing, search engine marketers need to devote a good deal of time staying on top of the SEO industry and its trends.

Today very few (and mostly inexperienced) optimizers / marketers would use spam methods to achieve high ranking. In many cases, spamming and the so-called "black-hat" SEO is recognized by automatic spiders as they become more and more intelligent. Even though we describe major spamming methods in the end of this course, we do it only in the sake of your awareness. We do NOT recommend using them, as there's no guarantee at all they can help, whereas it's very possible that your Web visibility can be seriously damaged as a result. Besides, it's just the wrong thing to do.

Search engines have already developed into sophisticated systems, and no doubt that development will continue, with improved capabilities for indexing pages deep within the site (many links away from the initial page). Also, the ability to handle dynamically generated pages (e.g. shopping carts) is expected to grow. Among other perspective trends are advanced non-HTML content indexing (such as PDF and graphics), improved ability to rapidly integrate new content such as news using XML feeds or other technology, organization of search results into logical categories (sometimes referred to as clustering), and other advanced features. For insights into the future of search, you could visit Google Labs (http://labs.google.com/). This is the beta area where Google showcases some of its upcoming technologies.

However, the traditional "highest-bidder" approach makes SEM (Search Engine Marketing) feel more and more like traditional print advertising. The Integrated Approach to SEM anticipates that the best marketing efforts will aim to leverage three principal components: paid advertising and analytics, on and off-site content optimization, and the qualitative enhancement of off-the-page factors . That is why we prefer to call the functions provided by Web CEO "Search Engine Marketing" rather than "Search Engine Optimization". When all is said and done, it is the traffic you get and the way this traffic converts that matters –even more than your site rank on a search engine. You can rank worse than your competitor, yet the percentage of your visitors that turn into buyers will be so high that you actually outperform your competitor several times.

1. Spider-based search engines have made their way from simple, spam-vulnerable algorithms to complex and sophisticated mechanisms which are dangerous to play with.
2. The Search Engine Optimization industry has developed a number of black-hat techniques to abuse the automatic site indexation and ranking performed by search engine spiders. These techniques are referred to as search engine spamming. Nowadays they can be considered neither legitimate nor effective.
3. Contemporary optimization techniques include the creation and optimization of site content, structure, quality and popularity.
4. Today's SEM is based on the idea that it's not the traffic itself matters, but how targeted and convertible it is.

Comments (0)

Subscribe to this comment's feed

Name

Email

Website

Title

Comment

Write the displayed characters

Add Comment