I am a relative newbie to website creation. Many webmasters (from beginners to experienced ones) continually scratch their head wondering how can I get Google to index my new content? Google has a program where you can automatically create an xml sitemap for your website. The sitemap contains a list of all your urls with the modification time and the change frequency for each url. After you have created your sitemap, you need to submit the sitemap to Google (using your Google sitemap account). After this, your need to ping (resubmit) Google every time your website changes. The idea here is that you are helping Google by letting them know you have new content on your site. The url list in the sitemap ensures that Google will not miss any of your urls while crawling your site.

Well there is the theory, how about REALITY?

The reality is that CREATING A SITEMAP AND PINGING GOOGLE WILL NOT GET GOOGLE TO COME TO COME AND INDEX YOUR SITE. What will happen? Several hours after your submit your sitemap to Google, Google will download your sitemap. Good so far. Then days will go by and Google will never come and index any of the urls in your sitemap. Boy that really stinks especially when considering it took me some time to figure out actually how to create a sitemap. Google does not make it real easy for you to create a sitemap. Google has a free script that you install on your web server that will automatically create and update your sitemap for you. The problem is that the script requires Python 2.2 to be installed on your web server. This seemed stupid to me since Python is not standard on the majority of web hosts (why not have the script written in PHP or PERL)? Anyway I found a free third party PHP sitemap generator at http://enarion.net/google/phpsitemapng/ that worked great.

In conclusion, having a sitemap that tells Google you have new content will not get Google to come to index your site. I guess the only way to get Google to index your site is to have incoming link from popular sites that Google can crawl along. Incidentally lack of regular crawling also makes the Google Adsense Search not work properly for searching your own site. I put a Google Search box on my site. However, because Google has not indexed my site yet, user searches of my site generate no results! This appears to be a problem for anyone who generates new content faster than Google indexes it. The user experience suffers due to this as I can not think of any reason a user would not what to find the most recent information on your site. In all fairness I should say a sitemap and Google Adsense search may be valuable for popular, well established websites that get indexed frequently. However, if you are relatively new to the online game, spend your effort on other aspects of your site!