Time and again, keywords suddenly ranked lower for a short time on individual days A look into the Google Search Console At first we thought it was an error in the tool, but the ranking deterioration and partial failures could also be observed in the Google Search Console: Screenshot GSC ranking With this keyword you could clearly see that the ranking was gone or significantly worse on individual days. We then compared the hiccup data with the these hiccups crawling statistics: It was noticeable that although there were always crawling peaks, these were often, but not always, related to the ranking deterioration. A look at the log files.
By now it was clear a log file analysis was due
These reveal two things to us. The crawling peaks from the GSC can also be found in the log files. Two URLs in particular are frequently crawled among the peaks. So it’s not a general increase in crawling, but rather it focuses on the top two URLs. Log files Two URLs in particular were crawled very Belgium WhatsApp Number Data frequently. The first was always returned by the bot with 200, but then returned 405. The second changed its status code several times: 2xx from 200 to 404, last response code in the log files 200, then 404 when retrieving. The two URLs are intended for Ajax requests , which update parts of the page. They shouldn’t actually be accessed directly by the crawler, so we blocked them via robots.txt. Number of URLs crawled Interestingly, there were also repeated peaks in the number of crawled.
URLs that were not analogous to the event
Peaks (this makes sense, because the event peaks only come from the two URLs mentioned above): peaks in log files There were peaks in the number of crawled URLs every 7 days. These URL peaks occurred Hong Kong WhatsApp Number List regularly every 7 days and primarily affected image cache URLs. We have often observed that Google crawls images every 7 days. However, it is typically the image bot, in this case it was the smartphone bot. I wouldn’t assign too much importance to that per se. There were often around 9 days between our ranking hiccups instead of 7 days . However, they often moved right around the dates of the URL peaks. The positive thing about it was that the rankings around these. Days were always only gone for a very short time. And some of them could not be reconstructed in the GSC.
