Reversing a 90% Traffic Collapse at a 650,000-Page News Publisher

In April 2026 we were called in to look at a news publisher that had lost almost everything. Discover, which had been carrying 82 percent of its traffic, was recording days of exactly zero clicks. Three Google updates had come and gone, everyone had a theory, and none of the audits had found anything.

The situation

The site runs around 650,000 articles, plus an AMP version of nearly every one, close to 1.3 million URLs in total. It is ad-supported, so traffic is revenue in a direct line. For years its dominant channel was Google Discover: about 139 million clicks and 1.7 billion impressions over a sixteen-month window, peaking near 780,000 clicks in a single day.

Dominant is the wrong word for it, in fact. Across the same window Web Search brought 26.6 million clicks and Google News 3.3 million, which puts Discover at 82 percent of everything. That ratio is the reason a single update scoped to one surface could take the business apart rather than dent it, and it is worth sitting with before reading anything that follows.

Between December 2025 and April 2026 the site lost about 90 percent of that revenue. The decline coincided with three separate Google events:

1. December 2025 core update. The first sharp drop coincided with its rollout.
2. February 2026 Discover core update. Scoped to Discover, and Discover was where the second drop landed.
3. March 2026 spam update, then the March 2026 broad core update. A spam update ran 24 to 25 March, the fastest rollout on record at under a day, immediately followed by a broad core update. The third drop landed on a domain already down two thirds.

By the time the March rollout finished on 8 April, the data was finally stable enough to diagnose.

The engagement began on 20 April 2026, after the third update had finished rolling out and with Discover collapsed to a flat line and recording days of exactly zero clicks, not zero as a figure of speech. Everything that follows was diagnosed on the wreckage, not predicted in advance of it.

A note on the word penalty, because it is what most publishers reach for when this happens. There was no manual action here and no message in Search Console. This was an algorithmic reassessment, and the distinction matters practically, not just semantically. With a manual action you have a specific issue documented in Search Console and a reconsideration request to file. With an algorithmic reassessment there is nobody to write to. The practical route back is to identify the signals that may be contributing to the loss, address them, and allow the site to be re-evaluated. That is slower, it is less certain, and it is why the diagnosis has to be right before anything gets touched.

Same content, no traffic

The obvious story wrote itself. It would have been easy to blame clickbait, prune low-quality articles, and move on when a news publisher with a heavy lifestyle tail loses traffic in core updates targeting sensational content.

That narrative was disproved by a single comparison. Although overall traffic collapsed, the proportion of content remained unchanged. The heavy emphasis on lifestyle content existed long before any update and was present during the domain's peak traffic periods.

A stable content mix does not prove what caused the collapse. It does make the content-type explanation much weaker, and that was enough to redirect the investigation toward sitewide signals: technical configuration, indexation, and link patterns. That redirection set the strategy for the whole engagement, and holding it proved the hardest part, because the publisher naturally inclined toward blaming the content he could see.

What the crawl found

A sitewide visibility loss does not announce its cause. Finding it meant crawling the site and reading what the CMS was actually doing to every outbound link, rather than working from a theory about what usually goes wrong. The audit found a link scheme operating in two layers.

Layer 1: the module whitelist, invisible to crawlers

The site used an External Links module that applied a nofollow attribute to every external link, sitewide, by default. Eight external domains, most of them commercial partners, were explicitly written into the module's exclude list. Excluded meant the module skipped its processing for those destinations entirely: no rel attribute, no target, no external-link icon. Those links shipped bare.

To Google they were ordinary followed outbound links, and because they were not qualified with rel="sponsored" or rel="nofollow", they were eligible to pass ranking signals. Nothing in the markup distinguished them from any other outbound link on the web. What made them a scheme was not any single link but the arrangement behind them, the same eight commercial destinations carved out of a sitewide rule on an estimated 330,000 pages.

Standard audit tools miss this. A crawler sees followed outbound links and reads them as normal, because normally they are. It cannot detect that on this site a followed external link was not normal at all, and that every one was an exception written into a configuration field. The policy lived in the CMS, not in the page markup, so the contradiction between the site's stated linking rule and its actual outbound links was invisible to any automated crawl.

The crawl did highlight the scale of the anomaly. A partial crawl covering 333,000 URLs revealed 171,000 pages linking to those eight domains, which projects to more than 330,000 pages affected at full scale. Meanwhile links pointing to government portals and official sources carried nofollow. Followed commercial links sitting next to nofollowed institutional ones is the inverse of a natural link profile. That discrepancy is what prompted the audit to inspect the module configuration underneath the markup.

The pattern was not confined to the archive either. A template block sat below the end of the article text, outside the article body field but inside the page, placing these links on articles sitewide. That distinction is why remediation needed two separate interventions, because the template block was one change while the links written into the editorial copy were thousands.
Why conventional audits missed it
A crawler could see the followed external links, their destinations, their anchor text, their placement, and how often they appeared. What it could not see was the module configuration, the exclude list, the commercial relationships behind those eight domains, or why those particular destinations were exempt from a rule that applied to everything else.

The finding came from joining three layers that are normally examined separately: crawl data, CMS configuration, and the commercial context behind the content.

Layer 2: the editorial content

Across the archive, the audit catalogued 4,390 outbound commercial links inside 2,184 articles. 87 percent were followed links with no sponsored or nofollow attribution, and 98.8 percent sat inside article body copy rather than in template elements, so no single template change could remove them. Some were embedded in hard-news pieces with no topical relationship to the commercial target, which is consistent with a link-scheme pattern rather than editorial linking, as with the geopolitical news story that carried a follow link to a partner voucher page.

One number in that set was worth more than the rest. Of the 1,943 pages carrying followed commercial links, 1,901 brought in no organic traffic at all. Ninety-eight percent of the pages holding the pattern were already earning nothing. That is consistent with the suppression seen elsewhere in the audit, though on its own it does not establish it, since advertorials often earn little organic traffic to begin with.

In December, the direct template-level affiliate links were removed and Google News partially recovered within weeks. That recovery was real enough to document at the time, and it was. It held until late March. The commercial incentives behind the pattern had not paused, though. In the weeks that followed, sponsored links returned to articles without disclosure and new advertorials kept publishing as before. By the time the February and March updates arrived, the pattern they evaluated was not a residue of the old scheme. It was active again, sitting on top of a layer the December fix had never touched, namely the module whitelist and the hub routing where article to intermediary hub to partner site reads to a crawler as the same pattern with one extra hop.

How the work was staged

The engagement ran as a gated sequence, not a single migration event. Each phase had to close before the next one could be trusted.

Phase 1: Forensic audit of the live site. Sixteen months of Search Console data rebuilt in BigQuery as the measurement baseline, a full crawl of the live site, and the two-layer diagnosis above. The audit also classified every article carrying commercial links into three tiers: genuine editorial that mentions a partner among other resources, mixed editorial-commercial, and de facto advertorial. The tier determined the treatment: attribute, disclose and attribute, or label and consider removal.

Phase 2: Remediation on the live site. The highest-impact signals did not wait for the rebuild. The module whitelist came out of the live site's configuration on 23 April, a five-minute change at the end of a forensic trail. The navigation cleanup followed the next day: the affiliate section link out of the header and footer, a latest-news section in its place. Editorial guidelines went to the newsroom at the same time, with a three-tier labelling rule and a one-question test for what counts as sponsored: would we have covered this piece without the commercial deal?

Phase 3: Content triage at scale. The classification produced execution lists, not opinions: 3,052 commercial and zero-value URLs marked for 410 Gone, 467 thin URLs for noindex, and 107 surviving advertorials queued for visible disclosure labels and rel="sponsored". That is 3,626 URLs individually decided. Keep or vanish was decided on data, organic positions and traffic per URL, checked against backlinks, not on how the content read. Nothing was mass-deleted, because removing authority alongside the problem is worse than the problem.

Phase 4: Pre-launch audit of the new build. The staging site was audited as a site in its own right, in parallel with the live one, and it caught regressions that would have shipped. The affiliate navigation link that had been removed from the live site was back in the new build's header and footer. The links module carried the same inverted configuration into the new build, so it was corrected in both directions there too. AMP was being retired, which put a surface carrying millions of clicks behind a single requirement, query-string 301 redirects, tested with curl before go-live rather than assumed. And an apparent total absence of NewsArticle schema on staging turned out to be the test tool blocked by basic authentication, a false alarm closed by verification instead of a panic ticket.

Phase 5: Executing the cleanup into a live core update. The plan was to ship the content cleanup with the new platform. On 21 May, Google began rolling out a core update, and the plan changed the same day: the 410 and noindex lists executed immediately on the live site, so the update would finish its re-evaluation against the clean domain instead of the old footprint. Conventional wisdom says freeze while an update rolls out. That is right for experiments and wrong for removals: if Google is re-scoring the site this week, the last thing to leave in front of it is the footprint you have already decided to withdraw. The rollout completed with the cleanup in place, one more week passed as a buffer, and only then did the new platform go live, on the afternoon of 10 June.

Phase 6: Post-launch. Sixty days of monitoring on the crawl observability pipeline described below, with the 107 editorial disclosure fixes drip-fed by the newsroom rather than shipped as one sitewide change.

Platform migration as a remediation event

The publisher had a Drupal 10 rebuild sitting ready in staging. Most teams treat a platform move as infrastructure maintenance. We used it as a remediation event.

• The launch preserved the cleaned state rather than creating it, with the 3,052 removed URLs still answering 410 Gone on the new origin rather than 404 or a redirect to the homepage, so the withdrawn footprint stayed withdrawn and the 467 noindex directives held.
• The 107 surviving advertorials carried over with their disclosure fixes queued, drip-fed after launch so the change did not read as a single sitewide event.
• The legacy article-path URL structure was mapped with single-hop 301 redirects to their canonical URLs on the new platform, preserving 3,247 backlinks from 128 referring domains. Stale mappings that pointed within the dead namespace were dropped rather than imported, since they would only have redirected into another 404.
• NewsArticle, Author, and Organization schema were enforced, alongside max-image-preview:large and a 1200px featured-image minimum, and publication timestamps were preserved so historical freshness signals were not reset on launch day. These are migration quality controls, not claims about what caused the recovery. Their job was to make sure the rebuild did not lose ground the old site already held.

One principle held throughout. A properly labelled advertorial is the opposite of the pattern that caused the damage. The problem was never sponsorship. It was undisclosed followed commercial links at sitewide scale. The sponsored assets were corrected with labels and attribution, not deleted in a panic.

Measuring instead of inferring: crawl observability for six dollars a month

The site ran on a commodity Cloudflare plan with no Enterprise log export and no access to raw server logs. We built the observability directly into the edge instead.

A Cloudflare Worker on the zone intercepts bot traffic at the edge and forwards verified hits asynchronously to a small GCP Cloud Function, which streams structured records into BigQuery. Requests claiming to be Googlebot are validated against Google's published crawler IP ranges rather than trusting the user-agent string, which anyone can spoof.

That distinction is not theoretical. Once the logs could tell the difference, roughly 53 percent of desktop "Googlebot" hits failed IP verification, 17,831 out of 33,432 in a single day, while smartphone Googlebot verified at 100 percent. The fake half was a scraper operation rotating through hundreds of one-shot residential IPs across nine countries, wearing the Googlebot user-agent, and being served full pages because the firewall trusted the string. 

Worth separating the two implications carefully, because they are not equally urgent. For indexing and rankings this changes nothing, because the smartphone crawler is the indexing path and it was clean. For bandwidth and content theft it is real and it was growing. Reported as an infrastructure observation, not folded into the recovery story, because it is not part of it. The BigQuery log tables sit in the same region as the Search Console bulk export, so crawl behaviour can be joined against ranking performance in plain SQL. The running cost to the publisher is about 6.50 US dollars a month.

What the logs caught

Within a day of launch it surfaced two things that would otherwise have been guessed at from indirect signals.

A broken legacy namespace, caught the day after go-live. The site's pre-2018 article URLs began returning 404 immediately after launch. The first one appeared in the logs at 09:39 the morning after go-live, and the rate climbed to over 4,000 hits a day. Verified Googlebot hit that one legacy path 19,727 times in the days that followed, which is what persistent re-checking of a high-authority URL that has just broken looks like from the server side.

It mattered because those URLs carried 3,247 live backlinks from 128 referring domains, several of them education-sector sites with real authority, making it one of the most linked legacy paths on the domain. The content itself was intact. Only the mapping from the old URL form to the new one had been lost, and the old slugs could not be mapped by hand because the new ones were transliterated and carried node IDs. The fix was to import the legacy alias mappings, with a single explicit redirect for the bare section path, then verify by curl that a sample resolved in one hop.

What the logs actually bought here was time, and it is worth being precise about that. Search Console would have surfaced these 404s eventually, with days to weeks of lag and sampling noise on top. The crawl pipeline compressed that into same-day visibility with before-and-after proof attached, which is what turned it into a vendor ticket that got fixed inside a week rather than a slow bleed nobody could date.

Asset protection. Logs showed verified Googlebot actively re-crawling legacy PDF documents and archived reference material under the files directory. That turned a migration-checklist assumption into the measured fact that those backlinked assets had to survive the migration intact.

A false alarm killed before it became a ticket. A surge of image 404s appeared after launch and looked like breakage. The logs showed they were requests for old image derivatives missing their security token, which Drupal correctly rejects, not broken media on live pages. That finding never became a vendor ticket, because the discipline was to verify before escalating.

Two weeks in, it caught something larger.

A crawl trap being used as a spam vector. The site's internal search route generated unlimited indexable URLs through its query string, with pagination observed past page 33,625. Verified Googlebot hit that single route 81,657 times in about twelve days, on a page that produces no readers. Worse, the search parameter was being injected with spam, and those spam URLs were confirmed indexed under the publisher's own domain. For a site being re-evaluated for link quality, having Google index unrelated commercial spam on your hostname is precisely the wrong exhibit.

The robots.txt already carried a Disallow: /search/ line, which matched nothing, because the trailing slash means it misses both the bare path and every parameterised variant. Fixing it in the obvious order would have made things permanently worse. Robots.txt blocks crawling, not indexing, so disallowing URLs that are already indexed freezes them there. Google can never re-enter to read a removal directive, and a spam URL with an external link petrifies as a permanent URL-only entry.

So the sequence was fixed rather than the file, starting with an X-Robots-Tag: noindex on the whole route, curl-verified on both the bare path and a parameterised one, and the robots.txt disallow held back until Search Console confirmed the URLs had actually dropped out of the index. Slower, and the only order that works.

The logs then showed it working. Verified Googlebot requests to that route peaked at about 8,400 a day in mid-June and fell to 87 a day by the third week of July, a decline of roughly 99 percent. The fall was gradual rather than immediate, which is what it looks like when Google has to recrawl a route before it can act on a directive it has just been given.

All of these come from seeing crawl behaviour directly rather than inferring it from Search Console.

The same data answered a question most sites can only guess at, which is where Googlebot actually spends its time. In June, 65 percent of verified Googlebot requests went to paths outside the core content sections. By August that had fallen to 44 percent, with the core sections rising from 35 to 56 percent of all crawl activity. Part of that shift is the search route no longer absorbing thousands of requests a day, which is the point rather than a caveat, since crawl budget is finite and what you stop wasting goes somewhere. Shares are used here rather than totals, because crawl volume moves for its own reasons and the interesting question is proportion, not absolute size.

The results, without the hype

Recovery is read here as a sustained trajectory, not a single traffic peak. The comparison below is the most recent quarter against the previous one.

A caution on how to read that table. The previous quarter was the bottom of the collapse, so these are gains off the floor, not a return to the previous peak. And the timeline matters.

The whitelist and navigation cleanup landed on the live site on 23 and 24 April. On 21 May, Google began rolling out a core update, and the content cleanup executed into that window. The recovery ramp began on 22 May. Two different sets of fixes sat in front of that update, at very different ages. The configuration and navigation changes were four weeks old and had been crawled and reprocessed for a month. The content removals were hours old, making it unlikely that Google had recrawled and processed all 3,052 changed URLs before the ramp began. That makes the older layer the more plausible explanation for the initial rise, rather than the fresh removals. The removals mattered for a different reason, which is that the rest of the rollout evaluated the cleaned site rather than the old footprint. The migration went live on the afternoon of 10 June, after the rollout had completed and a further week had passed. The recovery was already underway before the new platform existed in production.

Two weeks after launch the recovery got an unplanned stress test. Google ran a spam update from 24 to 26 June, its second of the year, global and across all languages. A site two weeks into a new platform and six weeks into a fragile recovery from a commercial-link problem is exactly the kind of site that gets caught by one. This one was not, and the trajectory continued through the rollout without a break. Google stated that this particular update did not target link spam, so we draw no conclusion from that about the link remediation specifically. What it does establish is narrower and still worth having. The rebuilt site went through a live spam rollout and came out the other side unaffected. Our read is that the core update re-scored the cleaned domain, and that is what turned the line. The migration's job was different, to make the fix permanent, carry the clean state onto the new platform, and relaunch without regression, which the clean post-launch crawl data confirms it did.

Isolating the true authority signal

The largest number is not the most important one. Discover produced the biggest absolute volume, and Discover is the volatile, feed-driven channel. The recovered Discover volume is mostly lifestyle content, the same kind the February update targeted once already. It is upside, not foundation.

The signal that matters is Web Search, the surface where the structural damage showed. There the recovery is concentrated in the sections that carry the site's topical authority: public sector news, careers and recruitment, and social benefits. Lifestyle search stayed roughly flat over the same window, which makes a broad-based visibility recovery more plausible than another viral spike. Click-through rate rose alongside it, from under 9 to over 10 percent, and average position improved from 6.6 to 5.3 across the property. Volume can spike from one viral article. Clicks, impressions, click-through rate, and average position all moving together across the property is more consistent with broad-based recovery than with a single URL doing the work.

The ranking data agrees with the traffic data, which matters because they come from different places. Keywords in the top three positions went from 8,877 to 18,031 between late April and early June, and top-ten keywords rose 76 percent over the same window. Those figures are from Search Console. A third-party ranking tracker, sampling independently and with no access to the same pipeline, showed the same trajectory over the same period. Two sources (GSC and Ahrefs) that do not share a method agreeing on the shape of a recovery is worth more than either one alone.

The distribution behind those rankings answers the obvious objection. Positions one to three and four to ten both grew substantially, while the twenty-one to fifty band shrank by nearly half and the total keyword count rose only modestly. Pages climbed. New keywords did not flood in. That distinction matters, because a site can manufacture keyword growth by publishing more, and this site did not, since the same body of content simply moved up the results. Which is what a recovery of authority looks like, as opposed to a recovery of output.

There is one more signal, and it is the one we would look at first if forced to pick a single number. Indexed pages went from 324,211 at the end of May to 547,279 by mid-August, a rise of roughly 69 percent. Clicks tell you Google is ranking the site again. Index coverage tells you Google considers the content worth holding on to, which is the slower and more structural of the two. It also puts the cleanup in proportion: 3,626 URLs were deliberately withdrawn, and the index then grew by more than 220,000. The removals were never a volume exercise. They were about what the remaining pages were sitting next to.

Section by section:

• Public sector news, the largest of those sections, expanded roughly tenfold off the April low.
• Careers and recruitment held top-of-page positions for its primary category terms.
• Social benefits showed a steady upward trajectory in non-branded impressions.
• On the July 2026 exam-cycle peak the site recorded 222,095 Web Search clicks in a single day, its strongest of the sixteen-month window and well ahead of the 2025 exam peak near 170,000.

What made it work

1. Hold the harder diagnosis. Volume collapsed while the content mix stayed constant, so the evidence moved the investigation away from content and toward sitewide signals, however tempting the content story was.
2. Removing the visible links does not remove the underlying problem. Deleting direct links while leaving an indirect hub route intact keeps the same pattern with one more hop.
3. Audit below the markup. The links were visible in the HTML. The policy that made them a scheme was not. A crawler can count followed links, but it cannot know they are exemptions from a sitewide nofollow default; that knowledge lives in CMS configuration, and no crawl output flags it.
4. Do not freeze removals during a core update. Freezing is right for experiments and wrong for cleanups. If Google is re-scoring the site this week, withdraw the footprint you have already decided to withdraw, and let the update evaluate the clean version.
5. Decide keep or remove on data. Organic positions, traffic, and backlinks per URL, not on how the content reads.
6. Order matters when removing indexed URLs. Robots.txt blocks crawling, not indexing. Disallow an already-indexed URL and you freeze it there, because Google can never re-enter to read the removal directive. Noindex first, confirm the drop in Search Console, disallow second.
7. Buy measurement, not inference. Edge logging into BigQuery gave direct crawl visibility for a few dollars a month, caught real issues on day one, and killed a false alarm before it wasted anyone's time.
8. Read recovery on the surface that was hit. Discover is a welcome revenue bonus. Web Search and Google News visibility are the more meaningful indicators of sustained recovery.

Written by Thanos Lappas, who ran this engagement.