After a model change, the chart needs an explanation
Tobias Raddatz
Mention rate jumps even though the website has not changed. The model also changed during that period. Check the measurement conditions before reporting a marketing gain or loss.
A model change need not invalidate every observation, but it changes a condition that may matter to comparison.
Record the before and after
Note the available model identifiers, provider and date. Check whether questions, search mode or query counts changed too. Several changes may coincide, and a chart alone cannot separate their effects.
Read answers from both periods. Are they longer? Do they use different sources or ordering? These observations do not fully explain the figures but can identify the next useful check.
When parallel measurement is possible
A limited overlap using the same questions may show differences between the observed answers. This requires the old model to remain available. Record the scope and treat the comparison as a sample as well.
If parallel testing is impossible, keep the limitation visible. Annotate the change in the chart. Adjusting figures by an arbitrary factor would only create an appearance of comparability.
Explain the next report
Separate the result from its proposed cause. “The brand appears more often in the current panel” may be supported. “Our copy change caused it” needs additional evidence.
A new baseline can be recorded after the switch. Earlier work retains its value as the documented result of a previous measurement setup. The report explains why comparison between the two sections needs care.
More from the magazine
What teams say about KI Score
“We finally see week by week where our brand shows up in ChatGPT and Perplexity.”
“The score finally gave us one number the whole team actually understands.”
“Instead of guessing which prompts matter, we now have real data.”
“Within the first week we already saw which competitor was beating us in AI answers.”
“Setup took one afternoon, not weeks of onboarding.”
“Finally a tool that takes AI Overviews as seriously as classic SEO.”
Measured where your customers ask
Move your brand ahead in ChatGPT and co.
Set up in minutes, your first measurement included.