Methodology

How every number is made.

If you can't check a measurement, you can't trust it. This page describes exactly how Witness collects answers, detects brands and calculates each figure, including what it can't tell you.

1. Prompts

Each project has a panel of prompts written the way buyers type them: discovery (“best X for Y”), comparison (“alternatives to a named competitor”), evaluation and problem-solving questions. Drafts are generated from the category, competitors and personas, then reviewed by the customer.

Prompts never contain the brand being measured. A question that names you will usually mention you, which would inflate the result.

2. Collection

Every answer is labeled with how it was collected:

  • Captured from the app. A capture provider loads the engine's consumer interface, signed out and localized to the project's market, and returns the answer text, the sources displayed and a screenshot. This is the closest to what a real user sees.
  • Captured from the API. Where no capture provider is connected, Witness calls the engine's official API with web search turned on. API answers can differ from the consumer app in wording and in sources, so they are never presented as app captures.

Each prompt is asked several times per run (3, 5 or 10, set per project), because engines give different answers to the same question. Failed captures are kept for the record and excluded from every number.

3. Brand detection

Whether a brand is mentioned is decided by deterministic matching of the brand name and its listed aliases, on word boundaries, case-insensitive. No AI model decides whether you were mentioned, so the same stored answer always produces the same result and anyone can reproduce it.

Position is the order in which tracked brands first appear in the answer. Sentiment is the one judgment made by a model: it reads the sentence around each mention and labels it positive, neutral or negative.

Citations are taken from the sources the engine attached to its answer, never inferred from the text.

4. Metrics and error bars

  • Mention rate: answers that name the brand ÷ answers read.
  • Share of answers (per brand): answers naming that brand ÷ answers read. Brands can share an answer, so these don't sum to 100%.
  • Cited in (per site): answers citing that site ÷ answers read.
  • Citation share (per brand): citations pointing to the brand's own domain, subdomains included ÷ all citations in the period. Several citations can come from one answer, so its interval is slightly narrower than the truth.

Every change is also read against the field: Witness compares your change with your rivals' average change, and if the two can't be told apart it calls it a rising (or falling) tide, a category-wide shift rather than your win. Where the connector reports whether the engine searched the web, your mention rate is split into searched answers, which your pages can influence, and answers from memory, which move with how widely you're mentioned elsewhere.

Rates on small samples carry a sample warning: fewer than 10 answers is “too few” to read as a rate, and fewer than 30 or a margin wider than ±10 points is a “thin sample”. Even 10 runs per prompt leaves about ±5.6 points of average error (Graphite, 2026).

Every rate is shown with a 95% Wilson score interval. Unlike the textbook formula, it stays within 0 to 100% and behaves at small samples and at 0 or 100%. The ± figure is half the interval's width.

Period-over-period changes use Newcombe's method for the difference between two proportions. A change is called real only if its interval excludes zero; otherwise Witness says it's within the noise.

As a guide: with 10 answers the range is roughly ±25 points; with 200 it is about ±7; with 1,200 about ±3. Small differences need large samples.

5. Experiments

An experiment compares a test set of prompts (ones a change should affect) with a control set (similar prompts it shouldn't), each against its own baseline from the 14 days before the change.

The reported lift is the test group's change minus the control group's change (difference-in-differences), so a shift that affects every prompt, such as an engine update, isn't credited to your change. Verdicts:

  • Worked or Made it worse: the lift's 95% range excludes zero.
  • No clear effect: the range includes zero.
  • Running / Too early: under 4 weeks in, or fewer than 60 answers in the test group.

6. What this can't tell you

  • Personalization. Signed-in users with history may see different answers. Witness measures the signed-out baseline.
  • Traffic. A mention isn't a visit. When an assistant names you, people often type your URL directly, so analytics record it as direct traffic.
  • Guarantees. Answers are probabilistic. Witness never promises placement, and you should be wary of anyone who does.
  • Sponsored answers. Some engines now show paid placements. Where a capture provider reports them, those answers are flagged as sponsored.

See what the engines say about you today.