How Google crawls your site
November 25th, 2007 by Gabi AndreiGoogle crawls the websites using spiders. They crawls the net, find sites and add them to his index. Even if you add manually the site using Google Add Url this does not guarantee that your site will be indexed faster.
What can be done? Generate a xml sitemap file of your website. After you generated it, let Google knowing about it - submit it to Google. Your site will be crawled in few days, usually the next day. You will need for sitemap submission a Google account, you can create one in 2 minutes. More information you find at How do I submit a sitemap. Another way of letting Google, Yahoo and Ask to know about your site, is to put your sitemap in the robots.txt file. This is called SE sitemap autodiscovery. They will find it and spider it.
The following is a list of software I found useful for sitemap generating:
Google Webmasters - Sitemaps Third Party Programs and Websites
Very useful for small sites (<500 pages) is XML-Sitemaps.com It is very easy to use, it generates the xml, txt, ror, html sitemap files.
A very useful blog related to Google crawling you can find at Google Webmasters Blog.
August 11th, 2008 at 11:30 pm
Monday I was searching for sites related to Search Engine Placement and specifically search engine optimization and found this site.
August 30th, 2008 at 12:30 pm
Submit Links to Social Bookmarking Websites Automatically! Get Thousands of Visitors! Increase Website Traffic and Income with Automatic Social Bookmarks Submitter. Prepare to upgrade your servers today