Essay ·
Synthetic users you can trust
Every research team is about to be offered the same deal: an AI persona that answers instantly, never needs recruiting, and costs nothing per session. Most teams will take the deal without reading the terms.
The terms are the problem. A large language model will role-play any user you describe, fluently, confidently, and with no necessary relationship to any human being who has ever used your product. Ask it whether "Sarah, a busy regulatory affairs manager" would use your new feature, and it will answer. It will answer well. And the answer will be a mirror of your prompt plus the internet's average intuitions, which is to say, a mirror of exactly the assumptions research exists to check.
An ungrounded synthetic persona isn't a cheaper research tool. It's a liability with a friendly face, because it launders guesswork into something that feels validated.
The question that separates tool from liability
There's one question to ask of any synthetic user: "who says?"
When the persona claims users struggle with X or would never adopt Y, who says? If the honest answer is "the model's training data, shaped by the prompt", you have a brainstorming aid at best. If the answer is "participant 7, in the interview on March 4th, and here's the passage", you have something new: research that answers back.
That difference is buildable. It requires three commitments.
1. Ground every persona in real participants. A trustworthy synthetic persona is a synthesis artifact, not a character sheet. Build it from actual participant dossiers (transcripts, empathy maps, journey maps) so the bot's voice is a composite of specific, real people. The archetype is the interface; it's the language product teams already speak: "would Sarah use this?". The participants underneath are the evidence layer.
2. Make provenance inspectable. Every claim the persona makes should be attributable to the participants behind it, down to the source artifact. If a stakeholder can't trace an answer to an interview, the answer is decoration.
3. Design the ignorance. This is the commitment most teams skip, and the one that matters most. A grounded persona must confess when its corpus doesn't cover a question ("my participants never discussed pricing") instead of improvising. Trust in a synthetic user comes less from what it answers than from what it declines to answer. An AI that admits the edges of its knowledge is exhibiting the one behavior that distinguishes evidence from confabulation.
And behind all three: evaluation. Audit the bots for paraphrase drift and fabrication the way you'd audit any research instrument for bias. On a schedule, with logged findings, before they face an audience.
What this is for, and what it is not for
Built this way, synthetic users are genuinely transformative for a narrow, valuable purpose: keeping existing research alive at the moment of decision. The PM's Tuesday-afternoon question no longer waits three weeks for a study. Nor does it get answered by whoever in the room sounds most confident. The knowledge your team already paid for, across studies and years, becomes something you can interrogate.
What they are not is a replacement for contact with users. A synthetic persona can only redistribute knowledge you already gathered; it cannot discover anything. The moment your bots become a reason to skip fieldwork is the moment they start amortizing stale truths into future mistakes. The practice that keeps them honest is the same one that created them: going back to real people, and refreshing the ground they stand on.
The uncomfortable part
The reason most synthetic-user deployments will fail isn't technical. It's that ungrounded personas are more pleasant than grounded ones. They always have an answer. They never say "my corpus doesn't cover that". They agree with the roadmap more often. A persona that confesses ignorance and cites its sources is friction. The productive kind, the kind actual users provide.
So the choice about synthetic users is really the old research choice wearing new clothes: do you want answers that are comfortable, or answers that are true? The technology is new. The integrity question is the one our discipline has always had to answer.
The system behind this essay is described in the case study.