The problem with traditional studies
Only large brands run studies regularly. Everyone else launches on gut feel, because traditional research is:
- Expensive: €5,000 to €50,000 per study (recruitment, incentives, analysis).
- Slow: 3 to 8 weeks between the brief and the results.
- Rigid: every concept variant is a new paid study.
- Biased: small samples, recruitment bias, social desirability.
The method, made accessible
The paper's core idea: decouple the production of an opinion from its measurement. We don't ask for a number. We measure the nuance of an open-ended response.
Why not just ask for a rating directly?
Asking an LLM to "rate from 1 to 5" produces degenerate distributions (everyone answers 4), high sensitivity to the prompt, and poor correlation with humans. The fix: let the persona express itself freely, then score that text from the outside.
- 1
1. Diverse personas
We generate a heterogeneous population of synthetic respondents (age, socioeconomic status, region, attitudes) to restore variance.
- 2
2. Natural-language response
Each persona reacts to the concept with a verbatim, in the language and culture of the simulated market.
- 3
3. Likert score by similarity
We compare the verbatim's embedding to anchor phrases (one per level); cosine similarity yields a continuous score and then a calibrated rating.
The anchor phrases (English example, purchase intent)
Scoring compares each verbatim to a versioned, language-validated set of anchors. Example on a 5-point Likert scale:
| Level | Anchor phrase |
|---|---|
| 1 | “I definitely wouldn't buy it, it doesn't interest me at all.” |
| 2 | “I'm unlikely to buy it, it doesn't really convince me.” |
| 3 | “I'm not sure, maybe, it would depend.” |
| 4 | “I'd probably buy it, it interests me.” |
| 5 | “I'd buy it without hesitation, it's exactly what I need.” |
The publication
"LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings" (arXiv 2510.08338, October 2025).
Our calibration
We don't ask you to take our word for it: we continuously measure the gap between our synthetic outputs and benchmark human data (public datasets, partner validations, internal hold-outs), and we display it.
- Synthetic ↔ human correlation
- 0.92
- Error on Top-2-Box
- 4%
- CI coverage
- 94%
- Concepts compared
- 26
The larger the validation base grows, the better and more defensible the method becomes. Call it a data network effect.
Indicative data, version showcase-v1
| Human | Synthetic |
|---|---|
| 1.4 | 1.5 |
| 1.7 | 1.6 |
| 1.9 | 2.1 |
| 2.1 | 2 |
| 2.3 | 2.5 |
| 2.5 | 2.4 |
| 2.6 | 2.8 |
| 2.8 | 2.7 |
| 2.9 | 3.1 |
| 3 | 2.9 |
| 3.1 | 3.3 |
| 3.2 | 3.1 |
| 3.3 | 3.5 |
| 3.4 | 3.3 |
| 3.5 | 3.6 |
| 3.6 | 3.4 |
| 3.7 | 3.9 |
| 3.8 | 3.7 |
| 3.9 | 4.1 |
| 4 | 3.9 |
| 4.1 | 4.3 |
| 4.2 | 4.1 |
| 4.3 | 4.2 |
| 4.5 | 4.6 |
| 4.6 | 4.4 |
| 4.8 | 4.7 |
Confidence intervals & reliability score
Every study displays a reliability index derived from the category's calibration density, the panel size (N → CI width), and the internal consistency of the responses.
Large sample: narrow confidence intervals, high reliability.
Mid-size sample: good reliability, keep margins in mind.
Small sample: indicative result, confirm on a larger panel.
A committee of models, not a single judge
Every synthetic respondent is drawn from a different model family. Mixing several AIs dilutes the bias specific to any single model — like a diverse human panel rather than one opinion.
- Claude
- GPT
- Gemini
- Grok
- Mistral
Less single-model bias
A single LLM imposes its style and training culture on the entire panel. Spreading respondents across five families (Claude, GPT, Gemini, Grok, Mistral) softens each one's idiosyncratic bias.
Less dependency
If one provider is unavailable or rate-limited, the study continues on the other models. No study depends on a single vendor.
Diversity and sovereignty
The pool includes a European model (Mistral) alongside US models, for genuine cultural and technological diversity.
Proven, not promised
Every answer records the model that produced it: the per-provider breakdown is traceable and verifiable.
In full transparency: this mitigates bias, it does not eliminate it entirely. We label it as such.
Credibility FAQ
Does this replace a real study?
It's a decision-support and prioritization tool, not a full substitute for fieldwork on very high-stakes decisions. Use it to screen concepts, pre-test a message or a package, and iterate fast, then keep fieldwork for the critical calls.
Why is it reliable if it's so cheap?
The low price comes from marginal compute (Haiku model, Batch API, caching), not from a compromise on method. The rigor is identical: semantic elicitation, calibration, confidence intervals. It's the production cost that collapses.
What about AI biases?
They exist (homogeneity, stereotypes, cultural or desirability bias). We mitigate them through forced persona diversity, prompt guardrails, localized anchors, and per-market calibration that corrects systematic skew. And we flag the cases where reliability is low rather than hiding them.