Google August Flux 2020
Semrush Sensor registered significant fluctuations in SERPs on August 11. Many webmasters noticed this and started talking about a new SERP update but John Mueller, in response to Barry Schwartz on Twitter, dispelled these assumptions:
As it became known later from the official statement of Google, it was a bug on their part and the problem was related to Caffeine. This is a crawling and indexing system that was introduced into the basis of the search algorithm back in 2010:
So, what is Caffeine?
Back in 2010, Google, one might say, completely changed the indexing system and adding data to the results. By introducing Caffeine as part of its core algorithm, the search engine made it possible to not only scan more information but add information in seconds, which made the Internet more relevant.
In fact, this was not an update of the search algorithm but a complete restructuring of it, taking into account the rapidly growing number of new resources and users on the Internet.
Scheme of the old and new indexing algorithm:
A few words for those who are not familiar with search technologies. When you search for information on Google, you are not working with a live web. The search is done on the web index created by Google. This index is very similar to the index at the end of a book, helping you find the information you need.
As Google described the new Caffeine indexing system:
“Our old index had several layers, some of which were updated more regularly than others. Most of the index was updated every two weeks. To update the index layer, we needed to analyze the entire network, which created a delay between the moment the page was found, and when it became available to the user.
Caffeine, in turn, allows you to analyze information on the Internet in parts and continually update our search index. This means that search results will display the most recent information, regardless of the time and place of publication.
The system processes hundreds of thousands of pages every second. If we imagine that these pages are paper, the stack of them would grow at a speed of about 5 km per second. Caffeine’s database contains about 100 million gigabytes of data and new information is being added at a rate of several hundred thousand gigabytes per day. You would need 625,000 iPods with the largest storage capacity to store this kind of data.
When creating Caffeine, we focused on the future of the Internet. This system is an excellent basis for building other, faster, and more voluminous indexes, as it can better adapt to the development of the network. More improvements coming this summer.”
So what happened on August 10th?
On the issue of the bug, check out Gary Illyes’ tweet:
The Caffeine indexing system performs several functions:
- Ingests fetch logs,
- Renders and converts fetched data,
- Extracts links, meta, and structured data,
- Extracts and computes some signals,
- Schedules new crawls,
- And builds the index that is pushed to serving.
And of course, he added that this is not a complete list of factors and it can be continued:
That is, in August 2020, we encountered a rather large error in the Google index, which affected all regions and niches.
On all thematic social platforms, webmasters complained about a sharp drop in traffic and a drop from the first to 5+ issue page. The big players took over the entire front page, crowding out local resources. Amazon, eBay, Shopify, and Pinterest were not affected, while smaller players completely dropped out of the SERP. It seemed that 2009 is back in the yard and Google is just starting its work on tracking search results:
Also, many webmasters at BlackHatWorld stated that after everything returned to normal on August 11, the sagging resources not only returned but also received a bonus in the form of additional traffic:
During the period of maximum fluctuations, these were sites with a low rating, spam, and affiliates that were not previously ranked on the first page, climbed into the top 3, but after the fix, they went into the shadows again:
BlackHatWorld
It’s hard to say what happened. A major bug that broke the world wide issue or update that went wrong. Most likely, this question will remain unanswered. Anyway, Monday, as usual, has become a stressful day for many webmasters around the world.
