With the full implementation of Big Daddy in the Google algorithm back in March, many have found Google indexing fewer pages which has webmasters trying to figure out how to get more of their pages indexed and found by searchers. Aaron Wall of SEOBook.com has written an article on the recent indexing phenomenon at Google entitled, “The Google Crawling Sandbox.”
It is a most excellent read that I will not repeat here. I would like to provide a few excerpts however and then encourage you to get a cup of coffee, get in a comfortable position and enjoy. It is well worth your time in understanding how Google might be changing the way they crawl the web.
Now a few excerpts.
Regarding the question of “what is the optimal size a site should be” Aaron states, “If you completely changing your site structure away from being usable to do things that might appease Google in a state of flux you are missing the real message they are trying to send. If you rely too heavily on Google then you might find they are in a constant state of being broken, at least from your perspective.”
On how authority systems influence the networks they measure Aaron says, “Search has a self reinforcing aspect to it as your links build up. A site with a strong history of top rankings gets links that other sites won’t get. Each additional link is a re validation of quality. The people at Google realize that they have a profound effect on how the web grows. Now that they have enough content to establish a baseline in most commercial markets they can be more selective with what they are willing to crawl and rank. And they are cleaning up the noise in their ad market as well.”
And one final excerpt – “If you just try to fit where Google is at today expect to get punched in the head at least once a year. If you create things that people are likely to cite or share you should be future friendly.”
Click here to read the full write up.