Synthetic research means asking an AI model to answer research questions the way a defined group of people would, instead of asking the people, or before you do. In about two years it has moved from academic papers to product menus, with research platforms selling synthetic panels that answer in minutes rather than weeks. The pitch is speed and cost; the evidence is more mixed. This guide covers what synthetic research is, where AI respondents have matched real people in published studies, where they break, and how to use them for ad and brand decisions without being misled.
What synthetic research is
- Synthetic respondents (synthetic panels) — a language model answers as a persona: an age, a location, an income, a set of attitudes. Qualtrics, for example, launched Edge Audiences in March 2025 for questioning synthetic respondents that represent target customers.
- Silicon samples — the academic term from Argyle and colleagues (Political Analysis, 2023), who prompted GPT-3 with the backstories of real survey respondents.
- Digital twins — models built from data about specific real people. Ipsos and Stanford announced digital twin panels on Ipsos KnowledgePanel in 2025.
The distinction matters: a twin built from a long interview with a real person is a very different object from a persona described in two sentences.
Where AI respondents matched real people
- Vote choice — Argyle et al. found correlations of 0.90 to 0.94 between GPT-3’s simulated votes and the American National Election Studies for 2012, 2016 and 2020, with a weaker fit for pure independents.
- Willingness to pay — Brand, Israeli and Ngwe (Harvard Business School working paper, 2023) got GPT-3 to value fluoride toothpaste at $3.40, against $3.27 in a human study, and simulated shoppers switched from Colgate to Crest when Colgate’s price rose. Other categories were less tidy: wine broke the textbook rule of diminishing marginal utility.
- Behavioral experiments — Horton (NBER, 2023) re-ran several classic economics experiments on a language model and got qualitatively similar results.
- Interview-based twins — Park and colleagues (2024) built agents from interviews with 1,052 people. They reproduced those people’s General Social Survey answers 85% as accurately as the people reproduced their own answers two weeks later. Demographics-only agents did worse.
The pattern: synthetic respondents do best on averages and directions, for populations and categories that are heavily represented in the data the models learned from.
Where they break
- Too little disagreement — Bisbee and colleagues (Political Analysis, 2024) found ChatGPT’s group averages close to real survey averages, but the spread of answers almost an order of magnitude smaller. 48% of the relationships they tested differed significantly from the human data, and about a third of those pointed the wrong way.
- Missing and flattened groups — Santurkar et al. (ICML, 2023) measured the gap between model opinions and the US public as comparable to the Democrat-Republican divide on climate change, and telling the model to answer as a specific group did not close it.
- Question-format artifacts — Dominguez-Olmedo, Hardt and Mendler-Dünner (NeurIPS, 2024) showed answers driven by the order and labels of response options; once corrected, responses drifted toward random.
- Instability — in the Bisbee study, small wording changes shifted the results, and the same prompt gave different distributions in April and July 2023 as the model was updated.
What this means for testing ads
An ad rarely fails on the average. It fails because it splits the room, lands with one age group and irritates another, or is new enough that little like it exists in the training data. Those are exactly the weak spots: compressed variance hides polarization, poor subgroup fidelity hides who you are losing, and novelty is where models fall back on the familiar.
So divide the labor. Synthetic and AI reads are useful for screening many variants, catching clarity problems and sharpening hypotheses before a concept test. They are a poor basis for a final go/no-go on a large media budget, for estimating response rates or for claims about specific segments.
A checklist before you trust a synthetic panel
- Benchmark against people — compare spread and subgroup results with a human sample, not just averages.
- Reword and rerun — if a small change in phrasing moves the result, treat it as noise.
- Log the model and the date — rerun key questions over time to catch drift.
- Randomize answer order — option order alone can tilt answers.
- Decide in advance what needs human confirmation — some decisions are too expensive to make on synthetic data alone.
What the industry bodies say
The UK’s Market Research Society warned in a May 2024 report that synthetic participants can produce normative results that flatten diverse groups. The ICC/ESOMAR International Code, revised in 2025, added responsibilities around AI, synthetic data and synthetic personas, stressing transparency and human oversight. Adoption is running ahead of the guidance: in Qualtrics’ own 2025 survey of 3,198 researchers, 69% said they had used synthetic responses in the past year.
AI reads plus human panels
The studies point to the answer Israeli, one of the HBS authors, gave: synthetic data is no reason to stop talking to customers. AdTest.AI follows that logic. AI scoring rates each creative across 13 dimensions in about 90 seconds, which is the right tool for iterating. Shortlisted ads then go to a real audience panel of 100 to 300 people, with results in 24 to 48 hours. Where the two disagree is often the most useful finding, which is why we combine them into a Mixed Score. For the wider set of methods, see our creative testing guide.
FAQ
Can synthetic respondents replace a human panel?
Not on current evidence. Averages often land close to real data, but studies find compressed opinion spreads, unreliable subgroup results and sensitivity to prompt wording and model updates.
How accurate are AI digital twins?
It depends on what they are built from. Twins built from in-depth interviews matched people’s survey answers 85% as well as the people matched themselves two weeks later; demographics-only agents did worse.
Are synthetic research results stable over time?
Not necessarily. The same prompt has produced different results as the underlying model changed, so record the model version and rerun important questions.
Is there an industry standard for synthetic data in research?
The ICC/ESOMAR International Code, revised in 2025, covers synthetic data and synthetic personas and stresses transparency and human oversight.
Study results describe specific models, prompts and populations at the time they were published; newer models may behave differently. Vendor figures are the vendors’ own claims.
