Synthetic user panels in qualitative research — what they're good for

Researchers using LLMs for synthetic users tend to fall into one of two camps. The first asks "can I replace real participants?" — and gets a disappointing answer. The second asks "where do synthetic participants belong in my pipeline?" — and gets useful work out of them. This post is for the second camp.

What synthetic panels are good for

Discovery research at speed. You're three weeks into a new problem space, you don't yet know what questions will produce useful answers from real participants, and you can't afford to burn a real recruiting cycle running the wrong interview. A 5-persona panel asked your draft questions in 20 minutes will show you which questions surface texture and which produce flat agreement.

Concept testing for asymmetric upside. A landing-page draft, a pricing tier, a feature name. You're not trying to validate the concept (real users do that); you're trying to invalidate fast. Five diverse personas will reliably find the flat spots in your pitch — the parts where every persona answers the same generic way, which is a sign that you've written something too vague to react to.

Longitudinal continuity testing. Real participants don't reliably show up to a re-interview six weeks later. A synthetic panel does. For studies where you're checking "do the same questions produce the same texture across time" — useful before scaling — synthetic panels let you run the longitudinal study before the actual longitudinal study.

Question refinement. Your interview guide has 20 questions; you can only ask 8 in a real session. Run all 20 against the synthetic panel; promote the 8 that surface the most divergent answers across personas; demote the ones that produce convergent fluff.

What they're bad for

Statistical claims. Even with 50 synthetic personas, you don't have a sample. You have 50 LLM completions conditioned on five personas with one underlying training distribution. There's no generalizability claim to make.

Emotional response. Real grief, real frustration, real delight register differently from LLM-completion proxies. Synthetic personas report emotions; they don't have them. Anything where the felt experience matters — bereavement, financial stress, parental joy — stays with real participants.

Purchase decisions. Synthetic personas don't have credit cards. They can rationalize price sensitivity; they can't enact it. If your study is "would you pay $20/month for this?", real participants are non-optional.

Edge cases. The 5% of real users who hold the system together by behaving in unexpected ways — the rebel who finds workarounds, the incompetent who needs handholding, the gamer who breaks the spec — don't show up in synthetic panels. The panels regress to the prescribed mean.

A pattern that works

1. Define the demographic + psychographic spread you want represented. Five personas is the working point.
2. Generate the panel with Moonborn. Audit pairwise distinctiveness; if any pair scores < 0.30, re-brief and regenerate that pair.
3. Script the interview guide with 15–20 questions. Mix open-ended ("walk me through...") with concrete ("what specifically would make you trust this?").
4. Run the panel with a fresh chat session per persona. Log drift scores; toss replies that drifted out of character.
5. Aggregate and theme in your own qualitative tool. Promote the high-divergence questions to your real interview guide; ship that guide to real participants.
6. Real participants answer the sharpened questions. Their answers are the data; the synthetic panel's were the hypothesis-sharpener.

What Moonborn provides

• Distinct personas with auditable diversity (pairwise distinctiveness > 0.30).
• Drift detection per reply, so a persona that slips out of character flags itself.
• Long-term memory, so a panel revisited weeks later remembers the prior conversation.
• Voice fingerprint persistence, so the same panel is the same panel three months later.

What Moonborn does not provide

The research instrument. Moonborn produces the responses; you bring the methodology, the coding, the theming, the validation against real participants, and the final claims. The panel is a tool, not a study.

Next

• Synthetic user research use case.
• Synthetic user panel tutorial.