# Worked AI-search measurement readout

All numbers below come from the fictional version-1 packet. No model, campaign or real analytics property was used. Read `README.md` for the declared rules before interpreting the CSVs.

## 1. Show completion before citations

| Wave | Planned slots | Complete | Error | Missing | Completion coverage | Cited completed answers | Citation rate among completed answers |
|---|---:|---:|---:|---:|---|---:|---|
| October 15 baseline | 6 | 5 | 1 | 0 | 5 / 6 = 83.3% | 2 | 2 / 5 = 40.0% |
| October 22 follow-up | 6 | 5 | 0 | 1 | 5 / 6 = 83.3% | 4 | 4 / 5 = 80.0% |

The completed-answer rates differ by 40 percentage points, but they do not contain exactly the same slots. The baseline lacks P03/trial 2; the follow-up lacks P02/trial 2. Matching only by the count of completed answers would hide that difference.

The four pairs complete in both waves are:

| Prompt/trial | Baseline citation | Follow-up citation |
|---|---:|---:|
| P01 / 1 | 1 | 1 |
| P01 / 2 | 0 | 1 |
| P02 / 1 | 1 | 1 |
| P03 / 1 | 0 | 1 |
| Total | 2 | 4 |

For this matched subset, the rates are 2 / 4 = **50.0%** and 4 / 4 = **100.0%**, a difference of **50 percentage points**. The matched subset covers four of the six planned slots, or 66.7% of the panel. These are descriptive results for a small chosen panel, not a market-wide citation share or a reliable expected result for the engine. Missingness may bias the subset. Matching trials does not establish that the guide change caused the difference.

The error and missing citation fields stay blank. A completed answer without a target link has a real zero. If all trials failed, the citation rate would be unavailable even though completion coverage would be 0%.

## 2. Apply the session window and exclusions

Start with all 13 raw rows and retain these exclusions in your record:

| Session | Reason excluded |
|---|---|
| S10 | Marked QA traffic, even though its source looks like an AI referral |
| S11 | Staging hostname, not the declared production hostname |
| S13 | Starts at October 23, 00:00 in New York, the exclusive upper boundary |

S12 is included: `2026-10-23T00:30:00Z` is October 22 at 20:30 in New York. Filtering the UTC date string as though it were a New York calendar date would wrongly remove it.

Included sessions = 13 − 3 = **10**. They are S01–S09 and S12. This is a session count, not a user count; there is no cross-session user identity in this file.

## 3. Count events and sessions with events separately

| Recorded session source / medium | Included sessions | Apply-click events | Sessions with at least one apply click |
|---|---:|---:|---:|
| chatgpt.com / referral | 2 | 2 | 1 |
| perplexity.ai / referral | 1 | 1 | 1 |
| google / organic | 3 | 1 | 1 |
| (direct) / (none) | 2 | 1 | 1 |
| newsletter / email | 1 | 0 | 0 |
| linkedin / organic_social | 1 | 1 | 1 |
| Total | 10 | 6 | 5 |

Overall sessions with an apply click = S01, S03, S04, S06 and S09 = **5**. Overall session action rate = 5 / 10 = **50.0%**. The six raw events are not six applications: S01 alone contains two clicks.

The declared AI-source bucket contains S01, S02 and S03: **3 sessions**, **3 apply-click events**, and **2 sessions with an apply click**. Its session action rate is 2 / 3 = **66.7%**. Dividing the three events by three sessions would answer “events per session,” not “share of sessions with an action.”

The bucket represents 3 / 10 = **30.0% of included recorded sessions**. That is not the share of all real visitors influenced by AI. The two direct sessions remain direct; neither their source nor the panel observations prove an AI origin. Google organic sessions also stay separate because the supplied session dimensions do not identify a specific Google search feature.

Seven included sessions have a guide-to-jobs event: S01, S02, S04, S05, S08, S09 and S12. That is 7 / 10 = **70.0%**. Do not divide the five apply-click sessions by these seven and present a sequential funnel: some apply-click sessions have no guide-to-jobs event, and the table has no event ordering. A real funnel requires a declared sequence and suitable event data.

## 4. Keep Search Console alongside the analysis

The separate synthetic Search Console snapshot has 30 GenAI impressions, 120 overall Web impressions and eight overall Web clicks. These are platform observations, not prompt-panel trials or sessions.

- Adding 30 and 120 double-counts an overlapping reporting view.
- Eight divided by 30 is not an AI click-through rate because the click numerator is overall Web.
- Eight divided by 120 = 6.7% describes the overall Web click-through rate for that synthetic report scope, not the GenAI slice.
- The three Google organic sessions do not need to equal eight Web clicks. The sources use different measures, reporting rules and, here, calendar-day boundaries. No event-level join is supplied.

## 5. A defensible short recommendation

> In this fictional exercise, the target appears in two of four matched completed answers at baseline and four of four at follow-up. Only four of six planned slots are matched, with one error and one missed observation across waves. The pattern warrants further observation; it does not establish an effect of the content change.
>
> Separately, the filtered site dataset has ten recorded sessions. Three have the declared AI-source attribution; two of those contain an apply click. The three clicks in that bucket are not three applications. We cannot connect those sessions to the prompt trials or estimate total AI-influenced traffic from these inputs.
>
> Before expanding the work, inspect the failed/missed observations, preserve the original records, and predefine another comparable observation round. Verify event behavior and reporting coverage with marked QA. Re-read the same analytics window after normal processing and record revisions. A causal claim would need a suitable comparison design that addresses competing explanations; collecting another before/after snapshot alone does not provide one.

No supplied field establishes completed applications, interview outcomes, revenue or causal lift. If those outcomes matter to the decision, define the needed evidence and collection permission before reporting them.