Guides · 2026-10-22 · By VSNARY | Emmanuel Orta
How to tell if an SEO fix worked: compare fixed and untouched pages
A before-and-after chart cannot tell a fix from the season, an algorithm update or an ad campaign. Comparing the pages you fixed with the pages you did not, on the same site and over the same weeks, can. Here is how to do it with the search data you already have.
To tell whether a website fix moved search traffic, compare the pages the fix touched with pages on the same site it did not touch, over the same weeks. Take each group's average daily clicks for the 28 days before the fix and the 28 days after, work out each group's percentage change, and subtract the untouched pages' change from the fixed pages' change. The season, algorithm updates and your own advertising hit both groups alike, so the gap between them is the part those cannot explain. It is evidence, not proof, because the pages you fixed were not chosen at random.
Somebody fixes something on a website, traffic goes up the next month, and the fix gets the credit. It might deserve it. But traffic also rises with the season, with a search engine update, with a competitor's site going down, with a radio advert, and with nothing at all. A chart of the month before and the month after cannot separate any of those from the fix, because all of them happened in the same month.
There is a better comparison, and it needs no extra software: the pages you fixed against the pages you did not, on the same site, over the same weeks. This guide walks through it with the page-level clicks and impressions any search console already gives you.
Why a before-and-after chart is not evidence #
A before-and-after chart answers one question: did the number change? It cannot answer the one you care about, which is whether the fix changed it. Everything that moves a whole site moves the chart, and most of what moves a whole site has nothing to do with a fix.
| What moved traffic | Shows up in a before-and-after chart | Shows up in fixed vs untouched |
|---|---|---|
| The fix itself | yes | yes, as a gap between the two groups |
| Season (spring for gardeners, winter for furnaces) | yes, looks like the fix | no, both groups move together |
| A search engine update | yes, looks like the fix | mostly no, unless it targets the kind of page you fixed |
| Your own advertising or a mention in the press | yes, looks like the fix | mostly no, both groups get the lift |
| A competitor's site going down | yes, looks like the fix | mostly no |
| Random week-to-week noise | yes | reduced, because both groups are summed over many days |
The middle column is why agencies show before-and-after charts: they always find something. The right column is why the comparison below is worth the extra ten minutes.
The idea in one paragraph #
Pages on the same site share the same weather. If spring lifts the site, it lifts the fixed pages and the untouched ones. If an update hits the site, it hits both. So instead of asking how much the fixed pages moved, ask how much more they moved than the untouched pages did. Statisticians call this a difference in differences. You do not need the name, only the subtraction.
Step 1: write down exactly which pages the fix touched #
This is the step most people skip, and it decides whether the rest works. A fix to one page's structured data touched that page. A fix to a template touched every page using it. A fix to robots.txt, the sitemap or the server touched the whole site, and for those there is no untouched group to compare against; say so and stop here, because a comparison with no control is a before-and-after chart in disguise.
Write the list as paths, such as /services/tree-removal/, not as descriptions. You will need to match them against exported data.
Step 2: choose two windows of equal length #
Use 28 days before the fix and 28 days starting on the day the fix went live. Four weeks contains four of every weekday, so a Monday-heavy business is not flattered by an extra Monday on one side. Longer windows are steadier but take longer to arrive. Do not slide the windows until the result looks good; pick them before you look.
Search data arrives with a delay of a few days, and the most recent days are often revised. Wait until the after-window has ended and a few more days have passed before you export.
Step 3: export clicks and impressions by page for both windows #
In your search console's performance report, set the date range to the before-window, group by page, and export clicks and impressions. Repeat for the after-window. You now have two lists of pages with numbers beside them.
Use the page dimension, not the query dimension. Queries are shared across pages and cannot be split cleanly into fixed and untouched.
Step 4: sum each group in each window #
Mark every row as fixed or untouched using your list from step 1. Sum clicks for the fixed pages in the before-window, the fixed pages in the after-window, and the same two sums for the untouched pages. Divide each by 28 to get an average day. A page that appears in only one window still counts in that window; leaving it out quietly biases the result.
Step 5: compare the two changes #
Work out each group's percentage change, then subtract the untouched change from the fixed change. That gap, in percentage points, is your estimate of what the fix did.
| Worked example (illustrative numbers) | Before, average daily clicks | After | Change |
|---|---|---|---|
| Fixed pages (1 page) | 100 | 150 | +50% |
| Untouched pages (2 pages) | 250 | 250 | 0% |
| Difference | +50 points |
If the untouched pages had risen 30% over the same weeks, the same +50% on the fixed pages would be worth only 20 points, and most of the apparent gain would have been the season. That is the case a before-and-after chart can never show you.
Doing it in a spreadsheet #
Paste the before export into one sheet and the after export into a second. In a third sheet, list every page that appears in either export, one per row, with four columns beside it: clicks before, clicks after, impressions before, impressions after. A lookup against each export fills them; a page missing from one export gets a zero in that window, not a blank, because a blank is easy to drop from a sum by accident.
Add a fifth column that says fixed or untouched, filled from your list in step 1. Then build a small summary of two rows and four columns: the sums for each group in each window, each divided by 28. The two percentage changes and their difference are three more cells. Keep the sheet; the next fix uses the same layout, and comparing several fixes side by side shows which kinds of change tend to move traffic on your site and which do not.
One check before you trust the result: add up the clicks across both groups for each window and compare the totals with the totals your search console shows for the whole site over the same dates. If they differ by more than a few percent, the page export was truncated or filtered, and a group is missing pages. Fix the export before reading anything into the gap.
Run it on impressions too #
Clicks depend on where you rank and on how the result looks. Impressions depend mostly on whether you are shown at all. A fix that made a page eligible for more searches (a crawl problem, a canonical pointing elsewhere, a page that was not indexed) should show up first in impressions. A fix to the title or description should show up in clicks with impressions roughly flat. Running both tells you which kind of change happened.
The traps #
Site-wide fixes have no control. If everything was fixed, nothing was untouched. Report the before and after and say plainly that it is only that.
The fixed pages were not chosen at random. They were chosen because something was wrong with them, and whatever made them broken might also predict how they would have moved anyway. A page that was failing to load might have recovered on its own. This comparison controls for what hits the whole site, not for what is special about the pages you picked. State that caveat whenever you quote the number.
Small numbers swing. A page going from 2 clicks a day to 4 is +100% and means almost nothing. If the fixed group earns a handful of clicks, report the raw numbers beside the percentage.
Two fixes in one window cannot be separated. If you changed the titles and the structured data in the same week, the comparison measures both together.
Do not move the windows after looking. Picking the 28 days that make the result look best turns evidence into decoration.
What this tells you, and what it does not #
A clear positive gap on a reasonable number of clicks is good evidence that the fix helped. No gap means the fix did not change search traffic in four weeks; it may still have been worth doing, because many fixes protect against a loss rather than produce a gain, and some take longer than four weeks to show. A negative gap is worth taking seriously and checking page by page.
It never tells you that the fix caused the change in the way a randomised test would. It tells you the change is not explained by anything that moved the whole site, which is a much stronger statement than a chart and a much weaker one than proof.
If you are deciding what to fix first, how to measure AI visibility covers the other half of the question, and the public dataset shows how often each defect appears across the sites CrawlCheck has read.
The main product
Found this on your own site? We fix it for $500.
Scan free to see where you stand. The fix is one site, every finding implemented and re-measured, with a sealed before and after.
Questions this post answers
How do I know if an SEO fix increased my traffic?
Compare the pages the fix touched with pages on the same site it did not touch, over the same 28 days before and after. Subtract the untouched pages' percentage change from the fixed pages' change; the gap is the part the season or an algorithm update cannot explain.
Why is a before-and-after traffic chart not enough?
Because season, search engine updates, advertising and competitors all move a whole site in the same month as the fix, and a before-and-after chart cannot separate any of them from the fix.
What is a difference in differences?
The change in the group you treated minus the change in a comparable group you did not, over the same period. On a website, the treated group is the pages you fixed and the comparison group is the pages you left alone.
How long should the before and after windows be?
28 days each is a good default. Four weeks holds four of every weekday, and it is long enough to steady the numbers without waiting months. Choose the windows before you look at the result.
Can I use this for a fix to robots.txt or the sitemap?
No. Those fixes touch the whole site, so there are no untouched pages to compare against. Report the before and after and say it is only that.
Does a positive result prove the fix worked?
No. It shows the change is not explained by anything that moved the whole site. The fixed pages were chosen because they were broken, and that choice can predict how they would have moved anyway.
Related findings
Comments
Comments are read before they appear. Nothing is published automatically, and no account is needed.
Writing about this? Facts, live figures and marks — every number on that page is dated and traceable to a scan.