Wednesday, July 6, 2022
HomeSEOCase Examine: How the Cookie Monster Ate 22% of Our Visibility

Case Examine: How the Cookie Monster Ate 22% of Our Visibility

The creator’s views are fully his or her personal (excluding the unlikely occasion of hypnosis) and will not at all times replicate the views of Moz.

Final 12 months, the staff at Homeday — one of many main property tech corporations in Germany — made the choice emigrate to a brand new content material administration system (CMS). The targets of the migration have been, amongst different issues, elevated web page velocity and making a state-of-the-art, future-proof web site with all the mandatory options. One of many foremost motivators for the migration was to allow content material editors to work extra freely in creating pages with out the assistance of builders. 

After evaluating a number of CMS choices, we selected Contentful for its fashionable know-how stack, with a superior expertise for each editors and builders. From a technical viewpoint, Contentful, as a headless CMS, permits us to decide on which rendering technique we need to use. 

We’re at present finishing up the migration in a number of levels, or waves, to scale back the danger of issues which have a large-scale destructive impression. Through the first wave, we encountered a problem with our cookie consent, which led to a visibility lack of virtually 22% inside 5 days. On this article I am going to describe the issues we have been going through throughout this primary migration wave and the way we resolved them.

Establishing the primary test-wave 

For the primary test-wave we selected 10 search engine marketing pages with excessive visitors however low conversion charges. We established an infrastructure for reporting and monitoring these 10 pages: 

  • Rank-tracking for many related key phrases 

  • search engine marketing dashboard (DataStudio, Moz Professional,  SEMRush, Search Console, Google Analytics)

  • Common crawls 

After a complete planning and testing section, we migrated the primary 10 search engine marketing pages to the brand new CMS in December 2021. Though a number of challenges occurred through the testing section (elevated loading instances, larger HTML Doc Object Mannequin, and so on.) we determined to go stay as we did not see large blocker and we wished emigrate the primary testwave earlier than christmas. 

First efficiency overview

Very enthusiastic about reaching step one of the migration, we took a have a look at the efficiency of the migrated pages on the subsequent day. 

What we noticed subsequent actually did not please us. 

In a single day, the visibility of tracked key phrases for the migrated pages diminished from 62.35% to 53.59% — we misplaced 8.76% of visibility in in the future

On account of this steep drop in rankings, we carried out one other in depth spherical of testing. Amongst different issues we examined for protection/ indexing points, if all meta tags have been included, structured knowledge, inner hyperlinks, web page velocity and cellular friendliness.

Second efficiency overview

All of the articles had a cache date after the migration and the content material was absolutely listed and being learn by Google. Furthermore, we may exclude a number of migration threat components (change of URLs, content material, meta tags, format, and so on.) as sources of error, as there hasn’t been any modifications.

Visibility of our tracked key phrases suffered one other drop to 40.60% over the subsequent few days, making it a complete drop of virtually 22% inside 5 days. This was additionally clearly proven compared to the competitors of the tracked key phrases (right here “estimated visitors”), however the visibility regarded analogous. 

Data from SEMRush, specified keyword set for tracked keywords of migrated pages

As different migration threat components plus Google updates had been excluded as sources of errors, it undoubtedly needed to be a technical subject. An excessive amount of JavaScript, low Core Net Vitals scores, or a bigger, extra complicated Doc Object Mannequin (DOM) may all be potential causes. The DOM represents a web page as objects and nodes in order that programming languages like JavaScript can work together with the web page and alter for instance model, construction and content material.

Following the cookie crumbs

We needed to determine points as shortly as potential and do fast bug-fixing and decrease extra destructive results and visitors drops. We lastly received the primary actual trace of which technical purpose could possibly be the trigger when considered one of our instruments confirmed us that the variety of pages with excessive exterior linking, in addition to the variety of pages with most content material dimension, went up. It is crucial that pages do not exceed the utmost content material dimension as pages with a really great amount of physique content material is probably not absolutely listed. Relating to the excessive exterior linking it can be crucial that every one exterior hyperlinks are reliable and related for customers. It was suspicious that the variety of exterior hyperlinks went up identical to this.

Increase of URLs with high external linking (more than 10)
Increase of URLs which exceed the specified maximum content size (51.200 bytes)

Each metrics have been disproportionately excessive in comparison with the variety of pages we migrated. However why?

After checking which exterior hyperlinks had been added to the migrated pages, we noticed that Google was studying and indexing the cookie consent kind for all migrated pages. We carried out a web site search, checking for the content material of the cookie consent, and noticed our concept confirmed: 

A site search confirmed that the cookie consent was indexed by Google

This led to a number of issues: 

  1. There was tons of duplicated content material created for every web page as a result of indexing the cookie consent kind. 

  2. The content material dimension of the migrated pages drastically elevated. It is a drawback as pages with a really great amount of physique content material is probably not absolutely listed. 

  3. The variety of exterior outgoing hyperlinks drastically elevated. 

  4. Our snippets all of the sudden confirmed a date on the SERPs. This might counsel a weblog or information article, whereas most articles on Homeday are evergreen content material. As well as, because of the date showing, the meta description was lower off. 

However why was this occurring? In keeping with our service supplier, Cookiebot, search engine crawlers entry web sites simulating a full consent. Therefore, they acquire entry to all content material and copy from the cookie consent banners will not be listed by the crawler. 

So why wasn’t this the case for the migrated pages? We crawled and rendered the pages with completely different consumer brokers, however nonetheless could not discover a hint of the Cookiebot within the supply code. 

Investigating Google DOMs and looking for an answer

The migrated pages are rendered with dynamic knowledge that comes from Contentful and plugins. The plugins include simply JavaScript code, and generally they arrive from a companion. Considered one of these plugins was the cookie supervisor companion, which fetches the cookie consent HTML from outdoors our code base. That’s the reason we did not discover a hint of the cookie consent HTML code within the HTML supply recordsdata within the first place. We did see a bigger DOM however traced that again to Nuxt’s default, extra complicated, bigger DOM. Nuxt is a JavaScript framework that we work with.

To validate that Google was studying the copy from the cookie consent banner, we used the URL inspection device of Google Search Console. We in contrast the DOM of a migrated web page with the DOM of a non-migrated web page. Inside the DOM of a migrated web page, we lastly discovered the cookie consent content material:

Within the DOM of a migrated page we found the cookie consent content

One thing else that received our consideration have been the JavaScript recordsdata loaded on our previous pages versus the recordsdata loaded on our migrated pages. Our web site has two scripts for the cookie consent banner, supplied by a third celebration: one to point out the banner and seize the consent (uc) and one which imports the banner content material (cd).

  • The one script loaded on our previous pages was uc.js, which is accountable for the cookie consent banner. It’s the one script we’d like in each web page to deal with consumer consent. It shows the cookie consent banner with out indexing the content material and saves the consumer’s resolution (in the event that they agree or disagree to the utilization of cookies).

  • For the migrated pages, except for uc.js, there was additionally a cd.js file loading. If we’ve a web page, the place we need to present extra details about our cookies to the consumer and index the cookie knowledge, then we’ve to make use of the cd.js. We thought that each recordsdata are depending on one another, which isn’t appropriate. The uc.js can run alone. The cd.js file was the explanation why the content material of the cookie banner received rendered and listed.

It took some time to seek out it as a result of we thought the second file was only a pre-requirement for the primary one. We decided that merely eradicating the loaded cd.js file could be the answer.

Efficiency overview after implementing the answer

The day we deleted the file, our key phrase visibility was at 41.70%, which was nonetheless 21% decrease than pre-migration. 

Nevertheless, the day after deleting the file, our visibility elevated to 50.77%, and the subsequent day it was virtually again to regular at 60.11%. The estimated visitors behaved equally. What a aid! 

Quickly after implementing the solution, the organic traffic went back to pre-migration levels


I can think about that many SEOs have handled tiny points like this. It appears trivial, however led to a big drop in visibility and visitors through the migration. For this reason I counsel migrating in waves and blocking sufficient time for investigating technical errors earlier than and after the migration. Furthermore, protecting an in depth have a look at the location’s efficiency throughout the weeks after the migration is essential. These are undoubtedly my key takeaways from this migration wave. We simply accomplished the second migration wave at first of Could 2022 and I can state that to this point no main bugs appeared. We’ll have two extra waves and full the migration hopefully efficiently by the top of June 2022.

The efficiency of the migrated pages is sort of again to regular now, and we’ll proceed with the subsequent wave. 



Please enter your comment!
Please enter your name here

- Advertisment -
Google search engine

Most Popular

Recent Comments