Are you ready for your Caffeine fix? You’d better be!
Matt Cutts, the head of Google’s spam team, announced yesterday at SMX Advanced Seattle 2010 that Caffeine is officially live and shared some of the ways their new index is different from the old Google.
In the old days… Google would crawl the billions of pages on the internet, index them all and then update one of their data centers with all the information they gathered. They would continue to do this to each of their data centers over several days until everything was up to date. This was affectionately known as the Google Dance. The problem was that you might have to wait a bit before you could find your pages in Google index or you might see different results in the SERP’s depending on which data center you hit when you searched.
With the release of Caffeine, there is a new Sheriff in town. Instead of crawling the billions upon billions of pages on the internet first and then updating a single index, Caffeine can now crawl a document, process it and then immediately pushes it out to all of their indexes. This makes Google’s indexing a much more dynamic entity and allows the user to find information closer to “real time” then ever before.
In fact, Matt stated that Google’s entire index is about 50% fresher than it ever was before. NICE! So…, instead of waiting days or weeks for your new or updated pages to be indexed, once Google crawls your page they can almost immediately push it out to their engine for the world to see. Caffeine’s new way of indexing the web also make it easier for Google to scale up and grow their search engine with even more relevant results for their users.
According to the official Google blog post, “Caffeine lets us index web pages on an enormous scale. In fact, every second Caffeine processes hundreds of thousands of pages in parallel. If this were a pile of paper it would grow three miles taller every second. Caffeine takes up nearly 100 million gigabytes of storage in one database and adds new information at a rate of hundreds of thousands of gigabytes per day.” Now that’s a lot of data.
From my perspective, Caffeine is set to bring forth a new level of near-real-time search to the market they already dominate. I gotta say…. I LIKE IT!!!
Here’s Matt talking about Caffeine at SMX.
Steve Scott is the owner of the Tampa SEO Training Academy
