← Back to blog

How to design your first prompt set for AI visibility tracking

Design a prompt set from real customer sentences, balance four quadrants, start with 15–20, freeze four weeks. Sources, writing rules, freezing discipline, plus our set's first real revision.

How do you design a prompt set for AI visibility tracking? The whole answer, up front: collect sentences from your customers' actual words, balance them across four quadrants, start with fifteen to twenty, freeze for four weeks before revising. The prompt set is the foundation of every AI visibility measurement you'll ever run — mention rates and citation rates are all fractions over the question count, so every drift in the set breaks your time series. This article walks through a cautionary example first, then the quadrant structure, where sentences come from, five writing rules, freezing discipline — and the first real revision our own prompt set went through.

The prompt set quadrant: buyer journey × brand naming, what each cell tests and how many

A cautionary example first

Someone opens an AI engine and types "what's the best GEO tool?" — Monday, Wednesday, Friday, three different answers. Conclusion: AI recommendations are useless noise. Wrong diagnosis. The engine isn't broken; the instrument is. That prompt fails in at least three ways: "best" is a leading word and invites divergent answers; "GEO tool" is a huge category with dozens of candidates, so sampling variance dominates; and three asks in one week reads normal fluctuation as a trend. A wobbly instrument invalidates every reading it produces — whatever you do next is wasted.

The example earns its place because it states the first principle of prompt set design: a prompt set is a measuring instrument, not a wish list. Instruments need stability, reproducibility, and a consistent vocabulary. Every rule in the rest of this article serves one of those three.

Four quadrants, four different tests

Slice your questions along two dimensions: where the buyer is in their journey (still understanding the problem, or comparing options) × whether the prompt names you (your brand, or just the situation). Four cells, each testing something different:

Quadrant Prompt shape What it tests Starting count
Understand · unbranded "Why doesn't AI mention my brand?" Scenario gap: are you present when customers describe the pain 4–6
Understand · branded "What kind of company is X?" Echo check: how AI retells you, and whether it's right 3–4
Compare · unbranded "Are there tools that get my brand into AI answers?" Competitive presence when buyers actually shop 4–6
Compare · branded "X or Y — how do I choose?" Head-to-head positioning and defense 3–5

The mix hides a choice most people get backwards. They stack branded prompts to eighty percent because those answers contain their name and feel good. That inverts the logic: your absence from a branded prompt is news; your absence from unbranded prompts is the disease itself — customers who haven't found you don't say your name. So keep the unbranded half at fifty percent or more, and keep the compare region at forty percent combined, because those two cells are closest to money. Quadrant-level stats have a side benefit: the weakest cell tells you which material to build next, picking up exactly where the brand mention gap method leaves off.

Where sentences come from, by trust

The quadrants give structure; the sentences come from four places, ranked by how much you should trust them:

  1. Sales and support transcripts. Questions customers asked with money on the line are the most expensive corpus you own. Use them verbatim. Not enough volume? Mine emails, tickets, group chats for phrasings that repeat.
  2. Your search console query report. Real people's actual words, one row per query. Even single-digit impressions carry the phrasing — a claim we've verified on our own site; the case section below tells that story.
  3. Community questions. Search your category terms on Reddit, Quora, or niche forums; how other people organize a question is a ready-made prompt set with authentic voice.
  4. Competitor FAQs and comment sections. A question a competitor dares publish in their FAQ is a question they've verified gets asked. Borrow the question, never the answer.

The first two sources keep it real; the last two keep it fed. One filter runs over all four: does this sound like something your customer would actually type? If not, discard — regardless of how authoritative the source is.

The query report deserves two extra lines, because it's the noisiest source. Filter out three kinds outright: bare brand queries (they can't measure a gap), navigational queries for your own site, and typos or gibberish. Cluster the rest by meaning and keep one or two of the most complete phrasings per cluster — the differences between clusters are the information; the duplicates within a cluster are noise.

Four blank sticky notes in a 2×2 grid — settle the four quadrants first, then decide which sentence goes where

Five writing rules

Sentences collected, five checks before a line earns a slot in the set:

  1. Verbatim beats polished; jargon dies. Internal product names and industry abbreviations don't appear in anyone's search box — "our enterprise LLM visibility monitoring suite" will never be typed by a buyer.
  2. One question, one thing. "How is X, what does it cost, and how does it compare to Y?" is three questions wearing one coat; split them or you'll never know which part moved the answer.
  3. Mix short and long. Short prompts ("GEO tool recommendations") probe the engine's default impression; long, scenario-filled prompts mirror real consultations. All-short misses situations; all-long misses defaults.
  4. Every name, every variant. Keep one or two prompts per name form — official, common abbreviation, misspelling-prone variants — for your brand and competitors alike. Official-name-only sets forfeit every battlefield where people type the shorthand.
  5. No leading. "Why is X the best choice?" measures the model agreeing with you. Such answers fail the four-week comparison anyway — change the wording and they collapse.

After the five checks, count: fifteen to twenty questions, every quadrant covered. Then ask each one a final time — would a customer type this, word for word? Only yeses get in.

The rule people break most is the first, so here's a before/after pair to calibrate: "Is our LLM visibility monitoring suite good?" becomes "Are there tools that get my brand mentioned in AI answers?"; "What is the difference between GEO and SEO?" becomes "I've done SEO — do I still need GEO?" Rewriting moves in one direction only: from your organizational language toward the customer's typing habits. The reverse — "cleaning up" real customer quotes into proper grammar — re-pollutes the corpus you just collected.

Freeze for four weeks, then revise

The moment the set is final, the most counterintuitive discipline kicks in: nothing changes for four weeks. Reading a direction out of engine answers takes three to four weekly observations — the cadence the "how long does GEO take" timeline established — and changing a question mid-window cuts every curve you've drawn. At week four's close-out, revisions get their turn, with three rituals: a version number in the header; a written reason per change (which data point, which event); and new questions logged separately from old ones — new questions start a fresh window and never compare against old ones in absolute terms.

Outside windows, revisions follow events: a new product line, a new market, an unexpected competitor type surfacing in answers — add questions any time. Adding doesn't touch existing questions, so the window survives. Once a quarter, audit the whole set and retire the weakest performers, as judged by four weeks of data — not by how you feel on audit day.

One boundary worth drawing: the engine list and the prompt set are two separate frozen objects. Adding Perplexity to your engine rotation this week doesn't touch a single question — the same set gains a column. Engines grow horizontally, questions live vertically; freeze and revise each in its own direction. Freeze them together and every anomaly becomes unattributable: did the question change, or the engine?

Our prompt set's first real revision

Here's our own lesson. The first prompt set was written at a desk: four quadrants, a few questions each, sentences built from what we assumed customers would ask — and it felt complete. The flaw surfaced after we hooked the site up to Google Search Console. The first batch of real queries arrived — nine impressions, zero clicks, small enough to ignore — but the phrasing jumped out: "how to measure brand presence in AI search", "how AI visibility is measured" — passive constructions centered on brand presence, not one of which our assumed set contained. We thought people asked "how to improve AI visibility"; real people asked "how is AI visibility measured".

The revision followed the data: real queries went into the set one by one, and the measurement article synced its headings and FAQ to the same phrasings — it happened outside a window, the version jumped from v1 to v2, and the reason field reads "first GSC queries". Looking back, the revision's real value wasn't the handful of new questions; it re-ranked our mental list of where sentences come from: the search console report moved from backup source to first rank, alongside support transcripts.

What this means for you: your query report may hold only a handful of impressions, but each one is a real person's phrasing — one is not too few. Copy them into your candidate list now; by the time your set reaches its second window, they're ready-made revision ammunition. Collecting sentences doesn't wait for traffic — it waits for you to start copying.

FAQ

How many questions should a prompt set have?

Start with fifteen to twenty, covering all four quadrants, sized so one round fits in an hour; fifty is the practical ceiling for manual tracking, past that you need a tool. Optimize for stability: twenty questions that don't move for a year beat sixty that drift every month — a question's value is week-over-week comparability, not coverage.

Should my prompts include my own brand name?

Yes, but only partly. Branded prompts ("What is X?") test the echo — how AI describes you and whether it gets it right; unbranded prompts ("Are there any tools that…") test the real gap, because customers who haven't discovered you yet won't say your name. You need both halves; a set built from only one gives you half a picture.

How often should I revise the prompt set?

Freeze it for four weeks — one observation window — and change nothing inside a window. Revisions carry a version number and a reason for every change; new and old questions are logged separately and never compared as absolute values. Event-driven additions are fine anytime: new product line, new market, a new competitor type — add questions, don't swap them, and the window still holds.

Can I just have AI generate my prompt set?

As a draft, never as the final set. AI generates the internet's average phrasing — it lacks your customers' vocabulary and specific situations, and tracking visibility against average questions measures you against nobody. The right use: generate a draft, then verify every line against support transcripts, search-query reports, and community questions; it's normal to replace more lines than you keep.

Anything to watch when actually asking?

Three rules: new chat per question — follow-ups measure conversational momentum, not real retrieval; exact wording, not a single changed word; fixed time of day, because answer sampling fluctuates and only same-window readings compare. Follow-ups ("why do you recommend it?") are fine for exploration, but the log only takes the first, untouched answer.

The prompt set is the cheapest instrument in your measurement kit and the most commonly neglected: twenty stable questions and one frozen window do more for your data quality than any expensive dashboard. Build the instrument first; then mention rates and citation rates get a denominator, and the citation tracking log gets a skeleton. WalkGEO ships the prompt set, probing, and gap attribution as a weekly close-out routine; to use it off the shelf, pricing is the doorway.

Done reading? Run the same audit on your own site

Sign up, then let the agent scan every page and hand you a prioritized fix list. Your first audit is free.

Run a free site audit