How Google Failed To Index New Pages

Search Engine Land has a great example of the issue, where new content from CNN.com failed to be indexed by Google, even though the site is one of the most popular on the web.

The bug first appeared on the 19th of October and has also impacted a number of smaller sites resulting in traffic loss and concerns their sites had been penalised by Google.

We have even seen the issue impact our own blog with an article from the 21st October still not being indexed in Googles results:

After a number of threads on Googles webmaster help forums, a Google representative stepped in and confirmed the issue:

Just to be clear, the issues from this thread, which I have reviewed in detail, are not due to changes in our policies or changes in our algorithms; they is due to a technical issue on our side that will be visibly resolved as soon as possible (it may take up to a few days to be visible for all sites though). You do not need to change anything on your side and we will continue to crawl and index your content (perhaps not as quickly at the moment, but we hope that will be resolved for all sites soon).

Update: Google have now confirmed the issue has been fixed and indexing should now be returning to normal. Content from the 19th " 24th October may take a couple of days to be included, but there should be no issues with new content.

If youve seen this issue impact your site, let us know in the blog comments below.