Google Sitemaps

One of the most often asked questions I see on webmaster forums is about Google Sitemaps. In short, a Google Sitemap is a mechanism that allows Google to index your site with more accuracy. If you do not implement a Google Sitemap, Google will still index your site. So why should you setup a Google Sitemap?

Managing Google’s Bot

The main dilema is that you want Google’s bots to visit your site and index as much as possible, but after that you only need Google to index new content and not just waste bandwidth indexing previous content. The robots.txt file offers only limited options, it can’t tell bots to index only new content, so Google came up with Sitemaps to help webmasters make bots work more efficiently. Google’s Sitemap format makes it easy to tell the search engine bot, what your urls are, when they were last changed, and the frequency with which they change. This way the bot knows what it needs to look at and what it can ignore. This should help reduce crawling and make for more accurate indexing as well.

Sitemap Benefits

For websites that use non-engine friendly urls or sites that are brand new and have put a lot of content, the benefits of using a sitemap is that indexing will be more accurate. If your site is new and has no pagerank, a sitemap will not add your site any quicker than if you did not use a sitemap. The idea is not immediate search engine rankings, the idea is to have correct indexing, so when Google does add your site officially to its databases, all of your pages will be included correctly.

If you use WordPress, Google Sitemap Generator is a WordPress plugin that can create and manage your Google Sitemap. You can also create your own manually, just reference the Google’s Sitemap protocol.

Google Rolls Out Structure Changes

Forget Web 2.0, the real talk on the Internet right now has to do with Google’s massive datacenter changes, nicknamed by almost everyone as “Big Daddy”. Instead of just a normal pagerank update that everyone can see in their Google Toolbar, Big Daddy is essentially an update of everything Google, according to most industry insiders.

Google is testing a new data center infrastructure, a feat much bigger and comprehensive than an algorithm change. Dubbed “Big Daddy” both in the search marketing blogs and forums and by the friendly folks at Google, this new data center—still in shakedown mode—will reportedly add new ground-level capabilities into the Google search function and drive those powers deep into all the algorithms with which Google searches, studies and indexes the Web.

The more apparant thing that most webmasters will notice as Big Daddy rolls out is Google’s new spider. Looking over your logs in February you might notice this:

Agent:Mozilla/5.0 (compatible;Googlebot/2.1; +http://www.google.com/bot.html)

Google’s new bot is based on Mozilla code and so all that work you did on making your site more open standards friendly should pay off, especially if you check your site with Firefox or any other Mozilla based browser.

For a while now webmasters have been noticing the aggressiveness of Google’s new search bot, and in case you decided to discourage Googlebot 2.1, you probably should think about reversing this action quickly. The notion is that Google now uses this new bot first to spider sites. If you send Googlebot 2.1 away, Google will stop sending any other bots to your site period!

All this means that Google is stepping up and upgrading their datacenters to meet the challenges of newer competitors like MSN and Yahoo.

Google Sitemaps Adds Features

Other than Google Analytics, which came out this week, Google also made some changes to Google Sitemaps. The new features amount to essentially some feedback as to what are the most popular search terms for you site, and what if any errors some of the sitemap urls are generating.

In order to enable the new options, you must verify your site by creating a specifically named html file on your site. The new features are helpful, but may be considered very insignificant when compared to Google Analytics.

Google Analytics

Google just announced that they are going to providing another service, called Google Analytics. The new service allows webmasters to have free web stats based on Google’s prior Urchin Stats service. All you have to do is signup with Google on their Google Analytics site, and signin using your Gmail, Adsense, or Adwords account.

Once you go through the signup process, you get a simple javascript to add to your html pages.

For WordPress users dt’s Blog.thedt.net site has a Google analytics plugin that gives you a simple text box for you to paste the script code into. I had to correct the plugin code, because for some reason it stripped out the php lines, so if you cannot correct the code from dt’s site, feel free to copy the Google Analytics plugin code from our WordPress forum.

Another WordPress plugin that is easier to use is Cavemonkey50’s Google Analyticator plugin. Unlike dt’s plugin, Analyticator just asks you for the user account code, and not the entire script.

However how does Google Analytics compete against the other web stats packages out there, well Eric Peterson gives us some of his thoughts on Google giving away web stats.

Promote Your Local Business

Here is something which many small businesses probably do not know about, but should probably start using. It is Local Search. Even if your business does not have a website (which it really needs to nowadays), you can still make search engines like Google, Yahoo, and MSN work for you. Google even makes it easy, so even if you do not have a web site, you can still be one of the local businesses that Google recognizes.

First try out a local search on local.google.com and see for yourself. Type any city or location and then narrow it down further by adding the service you are looking for.

If you are interested in promoting your business with Google, you have to apply at the Google Local Business Center. You can read more about local search in general at http://local.google.com/help/faq_local.html.

See Who Is Linking To Your Site

Here is a simple way of finding out what pages on the internet link back to your site.

Bring up Google.com, then type in your url for your site.

Example:
www.mysite.com

And enter. when the the results come up, go to the bottom and click : Find web pages that contain the term…. This will bring up new results.

Notice at the top right it will say something like: Results 1 – 10 of about 580 for “www.mysite.com”.

Google does not show all results, just the most important ones it finds, or ones that it thinks are important. Go to the last page of your results and you will see a link that explains this. If you click on this all your results will be shown.

repeat the search with the omitted results included

For small web sites this actually works better than running a search for link: www.yoursite.com. The bad part with this method is that it also brings up your own site’s pages.