Case study
AI research you can interrogate
How do you scale a research practice when demand doubles, without hollowing out the thing that makes research worth having?
Context
Research demand across my portfolio doubled in a year. Products requesting research grew 114%, reaching 71% of the active portfolio. No research team scales linearly with that curve, and mine is deliberately small; I hired every member of it. Meanwhile, the questions never stop between studies. Product managers ask "would Sarah use this?" on a Tuesday afternoon, and the honest answer, "we'd need to schedule a study", arrives after the decision has already been made.
What we did have was two years of user truth already in the building: 200+ discovery interviews run across the portfolio since 2024, all following the same semi-structured, journey-based guide (role, workflow walkthrough, tools, pain points). All of it sitting in transcripts, answering nobody.
The problem
Synthetic users are the obvious answer and the dangerous one. A large language model will play any persona you ask, fluently, confidently, and with no relationship to your actual users. An ungrounded synthetic persona isn't a research tool. It's a liability with a friendly face. If my team was going to put AI-powered personas in front of product decisions, every answer had to be defensible: traceable to something a real participant actually said.
What I built, and the decisions inside it
I built the system myself: around 60 interrogable persona chatbots that PMs, designers and stakeholders can question directly, each one grounded in real participant research, answering only from its own corpus, with every claim attributed to the person who said it.
Archetype-first, not participant-first. The first instinct is to build a bot per interviewee. We course-corrected early. The unit your organization already thinks in is the archetype, the named persona on the wall. Each bot is an archetype synthesized from two to five real participants, so talking to it is talking to the cross-study knowledge layer, in the language product teams already use.
Provenance is the product. Underneath every archetype sit per-participant dossiers built from transcripts and derived artifacts (empathy maps, journeys, jobs-to-be-done). When a bot makes a claim, it can attribute it to the specific participants behind it, traceable to the source interview.
A confess-ignorance contract. Each bot's system prompt obliges it to say what its corpus doesn't cover. Ask it something its participants never discussed, and it tells you so instead of improvising. Trust in a synthetic persona comes less from what it answers than from what it refuses to.
Evaluation as a guardrail, not a demo trick. An eval suite and routing checks guard the grounding. Sanity audits probe each bot for paraphrase drift and fabrication before it faces an audience, and the findings are logged, not vibes-checked.
Rules for when not to build. No bot without an audited persona synthesis and accessible transcripts. No single-participant bots; that's a different, weaker pattern. No new bot when an existing one already answers the audience's question. Discipline about the negative space is what keeps the system honest.
Built to be repeatable. The whole build is codified as a six-phase runbook, from corpus collection through demo script, that produces a new grounded bot in about a working day. The web interface, Persona Studio, has an About page written for non-engineers, because the audience is product teams, not developers.
What happened
Product teams can now interrogate a persona at the moment a question arises, not weeks later, and the answers cite their sources. The practice scales without pretending to replace fieldwork. The bots are built from real research and keep pointing back to it, which is precisely why the researchers treat them as leverage rather than competition.
Strategically, this buys UX a brake on the feature factory that slows nobody down. When grounded user truth is the path of least resistance, one question away at any hour, teams reach for it before they reach for assumptions.
From a live session
A product manager asked a persona how export should work, and it declined to fake a consensus: its source participants diverged, so it gave both stances with attribution, flagged what its corpus could not support, refused to invent numbers, and closed with the one design direction the two actually shared. A question that used to mean hours of digging through transcripts and notes now gets a defensible, cited answer in a single exchange.
What I'd tell another design leader
- Grounding is the whole game. A synthetic persona without provenance to real participants isn't a lighter version of research. It's the opposite of research wearing its clothes.
- Design the ignorance, not just the knowledge. The confess-ignorance contract earns more trust than any answer the bot gives.
- Speak the org's existing language. We synthesize to archetypes because that's the abstraction PMs already reason with. Leverage lands faster when it doesn't ask anyone to learn a new vocabulary.
Related: the essay Synthetic users you can trust · the discipline this scales: Research as an operating system · the method its dossiers stand on: When the method becomes code