Top SEO Blog

Before a site appears in search results, a search engine must index it. An indexed site will have been visited and analyzed by a search robot with relevant information saved in the search engine database. If a page is present in the search engine index, it can be displayed in search results otherwise, the search engine cannot know anything about it and it cannot display information from the page..

Most average sized sites (with dozens to hundreds of pages) are usually indexed correctly by search engines. However, you should remember the following points when constructing your site. There are two ways to allow a search engine to learn about a new site:

- Submit the address of the site manually using a form associated with the search engine, if available. In this case, you are the one who informs the search engine about the new site and its address goes into the queue for indexing. Only the main page of the site needs to be added, the search robot will find the rest of pages by following links.

- Let the search robot find the site on its own. If there is at least one inbound link to your resource from other indexed resources, the search robot will soon visit and index your site. In most cases, this method is recommended. Get some inbound links to your site and just wait until the robot visits it. This may actually be quicker than manually adding it to the submission queue. Indexing a site typically takes from a few days to two weeks depending on the search engine. The Google search engine is the quickest of the bunch.

Try to make your site friendly to search robots by following these rules:

- Try to make any page of your site reachable from the main page in not more than three mouse clicks. If the structure of the site does not allow you to do this, create a so-called site map that will allow this rule to be observed.

- Do not make common mistakes. Session identifiers make indexing more difficult. If you use script navigation, make sure you duplicate these links with regular ones because search engines cannot read scripts (see more details about these and other mistakes in section 2.3).

- Remember that search engines index no more than the first 100-200 KB of text on a page. Hence, the following rule – do not use pages with text larger than 100 KB if you want them to be indexed completely.

You can manage the behavior of search robots using the file robots.txt. This file allows you to explicitly permit or forbid them to index particular pages on your site.

The databases of search engines are constantly being updated; records in them may change, disappear and reappear. That is why the number of indexed pages on your site may sometimes vary. One of the most common reasons for a page to disappear from indexes is server unavailability. This means that the search robot could not access it at the time it was attempting to index the site. After the server is restarted, the site should eventually reappear in the index.

You should note that the more inbound links your site has, the more quickly it gets re-indexed. You can track the process of indexing your site by analyzing server log files where all visits of search robots are logged. We will give details of seo software that allows you to track such visits in a later section.

Spider - a browser-like program that downloads web pages.

Crawler – a program that automatically follows all of the links on each web page.

Indexer - a program that analyzes web pages downloaded by the spider and the crawler.

Database– storage for downloaded and processed pages.

Results engine – extracts search results from the database.

Web server – a server that is responsible for interaction between the user and other search engine components.

Specific implementations of search mechanisms may differ. For example, the Spider+Crawler+Indexer component group might be implemented as a single program that downloads web pages, analyzes them and then uses their links to find new resources. However, the components listed are inherent to all search engines and the seo principles are the same.

Spider. This program downloads web pages just like a web browser. The difference is that a browser displays the information presented on each page (text, graphics, etc.) while a spider does not have any visual components and works directly with the underlying HTML code of the page. You may already know that there is an option in standard web browsers to view source HTML code.

Crawler. This program finds all links on each page. Its task is to determine where the spider should go either by evaluating the links or according to a predefined list of addresses. The crawler follows these links and tries to find documents not already known to the search engine.

Indexer. This component parses each page and analyzes the various elements, such as text, headers, structural or stylistic features, special HTML tags, etc.

Database. This is the storage area for the data that the search engine downloads and analyzes. Sometimes it is called the index of the search engine.

Results Engine. The results engine ranks pages. It determines which pages best match a user’s query and in what order the pages should be listed. This is done according to the ranking algorithms of the search engine. It follows that page rank is a valuable and interesting property and any seo specialist is most interested in it when trying to improve his site search results. In this article, we will discuss the seo factors that influence page rank in some detail.

Web server. The search engine web server usually contains a HTML page with an input field where the user can specify the search query he or she is interested in. The web server is also responsible for displaying search results to the user in the form of an HTML page.

Google today introduced Google Sites™, an application that makes creating a team web site as easy as editing a document. With Google Sites, people can quickly gather a variety of information in one place – including videos, calendars, presentations, attachments, and text – and easily share it for viewing or editing with a small group, their entire organization, or the world.

“Creating a team web site has always been too complicated, requiring dedicated hardware and software as well as programming skills,” said Dave Girouard, vice president and general manager of enterprise, Google. “Now with Google Sites, anyone can create an entirely customized site in minutes and invite others to contribute. We are literally adding an edit button to the web.”Creating and editing a set of pages in a Google Site requires no knowledge of HTML or web design skills. People can start a new page with one click. Adding content is as easy as clicking the edit button. Sharing is as simple as sending an invitation. All content is instantly searchable, and Google Sites is accessible through any web browser.

Anyone inside an organization can begin using Google Sites by signing up for Google Apps™ communication and collaboration services through Team Edition — without having to burden IT for support. After verifying their business or school email address, people can instantly invite others to join, or easily identify people within their organization already using Google Apps.

With Google Sites, people can create a wide variety of sites, such as:

* an intranet to centralize company information;
* a team site to manage a project;
* a profile site including an individual’s resume, areas of expertise, and goals for the quarter; and
* a virtual classroom to post homework assignments, class notes and other resources.

Google Sites is secure and scalable. Users have full control over who can own, collaborate and view pages, and view version history for each site. Google Sites is built to scale to any sized organization — from a five person start-up to a 50,000 person enterprise or university — and requires no hardware or software to buy, install, or maintain.

Additional features include the ability to:

* Embed content from other Google products, including YouTube™, Google Docs™, Google Calendar™, and Picasa™
* Upload files of any type
* Customize a site’s look and feel

Google Sites is based on JotSpot™ technology and available in the Team, Standard, Premier, and Education Editions of Google Apps. If your business or school doesn’t use Google Apps, please visit http://sites.google.com and sign up for Team Edition with your work or school email address. Existing Google Apps administrators can enable Google Sites immediately from the Google Apps control panel.

Yahoo! today announced that it has launched a new research lab in Haifa, Israel — its first in the region. The Yahoo! Research Israel Lab will be led by Dr. Ronny Lempel, a renowned information organization and retrieval expert who will report directly to Dr. Ricardo Baeza-Yates, vice president of Yahoo! Research.

The Yahoo! Research Israel Lab, which opens today, will focus on boiling down complex technology problems into simple solutions to change the game in Web search. As a demonstration of its commitment to next-generation search, Yahoo! recently opened Yahoo! Labs - Bangalore and appointed eminent scientist Dr. Rajeev Rastogi to head the new India lab. Yahoo!’s arrival in Israel furthers the company’s commitment to discovering new technologies that deliver compelling experiences on the Web.

“Search is still in its infancy,” said Prabhakar Raghavan, head of Yahoo! Research. “At Yahoo!, we are working on the hard core science that can lead to search experiences that are significantly beyond the current art.” He continues to say, “Ronny Lempel is a great addition to the world-class team that we have assembled to develop a new approach to Web search. His expertise in search technologies and ties to local academia will help us draw on the best talent and knowledge from across the region and strengthen our worldwide R&D efforts.”

Lempel previously worked at the Information Retrieval Group at IBM’s Haifa Research Lab, focusing on research and development for enterprise search systems. Prior to joining IBM, he received his BSc., MSc. and Ph.D. in Computer Science at Technion-Israel Institute of Technology. He has authored numerous papers and received several awards for his work on search engine technology, and has twice won the Best Presentation Award at the International World Wide Web Conference.

“Israel is fertile ground for incredibly talented technologists, researchers and engineers, and the Yahoo! Research Israel Lab provides the best opportunity to create the technologies that will underpin the next generation of search on the Internet,” Lempel said. “I look forward to building the Haifa team with the best talent this region has to offer.”

Google announced today that it has completed its acquisition of DoubleClick, a company that offers online ad serving and management technology to advertisers, web publishers and ad agencies.

Eric Schmidt, Google’s Chairman and Chief Executive Officer, said, “We are thrilled that our acquisition of DoubleClick has closed. With DoubleClick, Google now has the leading display ad platform, which will enable us to rapidly bring to market advances in technology and infrastructure that will dramatically improve the effectiveness, measurability and performance of digital media for publishers, advertisers and agencies, while improving the relevance of advertising for users.”

Search Engine Optimisation is not something that you can acheive with a few minutes work, it is an ongoing task. We hope that our articles and tools will provide a valuable resource in your quest to improve your sites position in the Search Engine Results.

The Page Rank Checker will look up the Google Pagerank of up to 10 websites at a time.

The Multi Datacenter PR Checker Lets you look up the PR of your website at all of Google’s Datacenters.

Find the Pagerank of all internal pages of your website with the Internal Pagerank tool. Here you can check the Pagerank of all the indexed pages of your website.

The Link Popularity will tell you how many sites link to you. It checks the results in AlltheWeb, AltaVista, Google, MSN and Yahoo Search Engines.

Find out how many Indexed pages are found in AlltheWeb, AltaVista, Google, MSN and Yahoo Search Engines for your website.

Check if your site has an ODP (DMOZ) Listing.

Use the Search Engine Results Pages tool to check how your site ranks for up to 10 keywords in the big 3 Google, Yahoo and MSN.

We have added a new Tool - The IP Address Checker It will look up the IP address for domain names which is invaluable when looking for link partners.

Designing a search engine friendly web site is not complicated, but it requires dedication and constant communication between the graphic designer, content developer and SEO analyst, as well as special attention to the site’s architecture. Site architecture should not be an afterthought, nor should SEO.

According to Bruce Clay, “Strategic Web Site Development is the development of an integrated technical and marketing design that will allow the Web Site owner to realize their objectives for the site.”

Your site must be both visually appealing to visitors and contextually appealing to the search engines. In order to attain both of these goals, we recommend enlisting a graphic designer to design the look and feel of the site, and a content developer/ writer/ editor to create a clear message for your site. This will cause the search engines to rank your site higher and will increase your visibility for visitors. Creating a site that is easily navigable, visually appealing, and gives your visitors what they are looking for, will encourage visitors to stay longer and come back often.

Keywords

We recommend there be at least 200 words as close to the top of your HTML code as possible. Within those first 200 words of content, you need to make sure that you are using keywords in a manner that makes sense to a visitor and will allow the search engines to “see” your site as a subject matter expert. Never stuff your pages with keywords (within the Meta tags or content) or the engines will penalize or ban you for spamming. Keyword research can be time consuming, and perhaps even daunting, but it is an important step in getting your site ranked. Some of the most important things for users to remember when they are doing keyword research are:

• Use a mixture of both broad and targeted words • Prepare a short list of words beforehand that your site is trying to promote and/ or words about your industry • Use the keywords properly

Shari Thurow, at ClickZ, says it well, “Write and design your site using the words your target audience types into search queries.” More information regarding keyword research can be found in the Choosing Keywords section of the Bruce Clay, Inc. site.

Once you have completed your keyword research, created your keyword list, and constructed your Meta tags, check that you have at least 200 words of succinctly written, grammatically correct content that tells your visitors what your page/ site is about. When this is finished, your site is ready for submission to the search engines.

Before submitting your site to the engines, we recommend you check it against Bruce Clay’s Quality Site Criteria test. This is a great outline of the steps that should be followed during the design process as well, not just when you are ready to launch/submit the site.

Site Architecture

While the keyword research is being done, your designer should be working on site templates and site architecture. Using CSS, IFRAMES, table tricks and external JavaScript will ensure that the search engine spiders are able to crawl your site without getting bogged down within the HTML code. You want them to be able to go to the page and quickly establish what your site is about. This will determine whether or not you get ranked.

During this time, your designer should be working with the content developer to ensure that the pages that are going to be submitted hold at least 200 words of unique and valuable content.

The content developer should be working closely with your SEO analyst to make sure that the keywords are being used properly throughout the pages/ site. This means using heading (h1, h2, h3) tags, and links within the content using keywords, anchor text, etc.

Search Engine Guidelines

A good place to look for ideas on Web design techniques is in each engine’s specific design guidelines.

Google says the following:

• Make a site with a clear hierarchy and text links.
• Offer a site map to your users with links that point to the important parts of your site.
• Create a useful, information-rich site, and write pages that clearly and accurately describe your content.
• Think about the words users would type to find your pages, and make sure that your site actually includes those words within it.
• Try to use text instead of images to display important names, content, or links.
• Make sure that your TITLE and ALT tags are descriptive and accurate.
• Check for broken links and correct HTML.
• If you decide to use dynamic pages (i.e., the URL contains a “?” character), be aware that not every search engine spider crawls dynamic pages as well as static pages.
• Keep the links on a given page to a reasonable number (fewer than 100).

MSN says:

• In the visible page text, include words users might choose as search query terms to find the information on your site.
• Limit all pages to a reasonable size. We recommend one topic per page.
• Make sure that each page is accessible by at least one static text link.
• Keep the text that you want indexed outside of images.
• Add a site map.
• Keep your site hierarchy fairly flat.

Yahoo! suggests:

• Build a quality site.
• Get big—sites with over 100 pages will attract more attention from search engines looking for sites with quality content.
• Say something—sites with frequently updated and new content rank higher in search engines typically.
• Use keywords not only as a tag.
• Exchange links.
• Get involved and get your name out there.

Site Optimization Tips

This excerpt from an interview with Laura Lippay, SEO Program Manager for Yahoo, is very much worth noting. If there were 3-5 site optimization tips you were to recommend to web masters, what would they be?

1. Usability comes before SEO – better yet, they should work hand in hand…
2. SEO isn’t just about H1 tags and title tags - more importantly, you need traffic…
3. You can listen to what everyone else preaches about what works for SEO or you can find out for yourself…

Conclusion

Designing a search engine friendly web site is not difficult, but it is time consuming. As long as you are prepared to be methodical; do careful keyword research; create unique, interesting, well-written content; and work closely with all the members of your team, you will be rewarded with high engine rankings and recognition in your field of expertise.

A list of the Top 100+ Search Engines of 2007

A9 amazon.com answerbag AOL Ask.com Ask.mobile askville AURA! Baidu bessed blinkx boing ChaCha ClipBlast! Clusty collarity CometQ d e c i p h o del.icio.us digg digg labs swarm eurekster exalead Feedster FINDITALL GIGABLAST Google Google /*Code Search*/ Google Mobile ICEROCKET inQuira ixquick Jambo Jyve KartOO keyCompete Kosmix krugle KwMap last.fm like Live QnA LiveDeal lurpo MavenSearch mnemomap MS. DEWEY mystrands nayio oodle Opera Mini Pagebull pluggd PreFound Quintura kids Quintura retrevo riya ROLLYO searchmash SearchTheWeb2 SEO Discussion Search Singing FISH Skweezer snap Sphere Sphider SPURL.net SpyFu SQUIDOO Srchr SurfWax Technorati thefind.com trulia url.com Vivisimo W a l h e l l o Webaroo WEBBRAIN What to RENT? whonu? WIKIO Windows Live wink WiseNut wondir Yahoo! ANSWERS Yahoo! MINDSET Yahoo! Mobile Yahoo! SEARCH Yahoo! Search Builder Yedda yoono yoople ZABASEARCH ZEBO Zillow.com Zippy ZOO.COM ZUULA