By Jennifer Horowitz (c) 2008 It happens sometimes. Here are some of the common reasons you may not be indexed: Already Indexed: The major engines won't tell you if you're listed; it's up to you to find out. The method to discover if a page or domain has been indexed varies from one engine to another. Don't assume you're not indexed just because you searched through keywords and you didn't come up in the first few pages of results. You could still be indexed and end up at the bottom of the heap. Free Sites: Because of all the "junk" submissions from free web sites like Geocities, many engines choose not to index sites from such domains or limit the number of pages they accept.
Index Time: It hasn't been indexed yet. The amount of time before the engine indexes your site should be listed on the search engine's submission page, but these aren't always accurate or may be out of date. On the average, index times range from one to eight weeks depending on the engine. Some engines like AltaVista and Inktomi provide paid options if you wish to be indexed more quickly.
TIP! Time frame and expectations: Allow up to 4 months, if you are number 10 and want to be number 1, then it may just be time that is needed - but if you aren't showing up at all, then you need to look at keywords, content, title, description and keyword tags.
Roadmap from Home Page: Some engines have been known to drop pages that cannot be traveled to from the home page. HotBot has been rumored to do this. Think of your site links as a series of roads from one page to another. If there's no road from your home page to the page you want indexed, a search engine may decide the page is unnecessary.
External Links: Some search engines like Google and HotBot have been known to refuse to index Web sites that don't link to any other sites. Or, they may index your home page but refuse to index any other pages unless there are links from another domain. Or, they may index you for a while but then "prune" their database later because you didn't achieve any external links after a certain period of time.
Frames: Content inside of HTML frames can cause problems with submissions because the search engine may index the main content of the page, but not the surrounding menu frame. Visitors to your site find some information but miss the associated menu. It's generally better to create non-framed versions of your pages.
Spider Blocks: Search engine spiders cannot index sites that require a registration or password, and they can't fill out forms. This also applies to indexing of content from a searchable database. The solution is to create static pages that the engines can find and index without performing a special action on your site. Depending on your database system, there are both utility programs and companies that can assist you with this.
Guilt Through Association: If your Web site shares the same IP address as other Web sites on your host's Web server, you may find your IP quietly banned because of something someone else did. Ask your hosting service if your domain name has its own unique IP assigned to it. If not, ask them to move it to its own IP to avoid being penalized because of someone else.
Dynamic Pages: Dynamic pages with URLs containing special symbols like a question mark (?) or an ampersand (&) are ignored by many engines. Pages generated on the fly from a database often contain these symbols. In this situation, it's important to generate "static" versions of each page you want indexed. Fancy scripts and code on a page can hurt your rankings. When it comes to search engines, simple is better.
Large Pages: If your site has a slow connection or the pages are very complex and take a long time to load, it might time out before the spider finishes indexing. To avoid this, limit your page size to 50K or less. A good rule of thumb is that: page size + cumulative image sizes on the page = 50K-70K. If it is greater than that amount, visitors with dial-up connections will leave before the page fully loads.
Unreliable Hosts: It pays to have a reliable hosting service. If your web site doesn't respond when the search engine spider visits, you won't be indexed. Even worse, if you are indexed and they pay a visit when your site is down, you could be removed from the database. Spam: If you use questionable techniques that might be considered an overt attempt at spamming (i.e., excessive repetition of keywords, same color text as background) an engine may ignore or reject your submissions.
Redirects: Redirects or meta refresh tags sometimes cause the engines to have trouble indexing your site. If the engines think you are trying to "trick" them by using "cloaking" or IP redirection technology, they may not index the site at all.
Proper Directory Submissions: When submitting to a directory site like Yahoo, Open Directory, LookSmart, and others, a live person reviews your site. They decide if the site is of sufficient "quality" before they líst it. These directories can help you get listed with other engines, so make sure you give your directory submissions the attention they need.
Page Limits: Search engines will only spider so many pages of your Web site. This could be a few dozen or three or four hundred depending on the engine. Google is one engine that tends to crawl deeper into your site. How deep they go may depend on factors like your link popularity. Sites with higher link popularity are deemed "worthier" of more thorough indexing.
Random Errors: Sometimes the engines simply lose submissions at random because of bugs and technical errors. Mistakes happen - remember, they're managing a database containing hundreds of millions of pages.
Jennifer Horowitz is the Director of Marketing and co-owner of EcomBuffet.com . Since 1998, her expertise in onlíne marketing and Search Engine Optimization (SEO) has helped clients increase revenue and achieve their business goals. Jennifer has written a downloadable book on Search Engine Optimization and has been published in many SEO and marketing publications. Jennifer can be reached at Jennifer@ecombuffet.com.
All Optimized and Going Nowhere?
Label: SEO
