How an AI reproduces purchase intent without ever asking it for a rating
By The Panelia team6 min read
The scientific idea behind Panelia, made accessible: why asking an LLM for a number doesn't work, and how semantic similarity restores realistic responses.
When you picture "using an AI to run a market study," you usually imagine a simple recipe: describe a product, then ask the model to rate its purchase intent from 1 to 5. It's intuitive, but it's also the wrong method. That's exactly what the research shows.
The trap of the direct rating
Asking a large language model (LLM) for a number produces degenerate distributions: everyone answers "4." Variance collapses, responses become hypersensitive to how the question is phrased, and the correlation with real consumers is weak. The model "overplays" certain ratings because it's optimizing for a plausible answer, not an opinion.
In other words: a number requested directly carries almost no usable information.
The core idea: decouple the opinion from its measurement
The paper "LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings" (arXiv 2510.08338) proposes an elegant indirection. Rather than asking for a score, it proceeds in two steps:
- The persona expresses itself freely. We have the model embody a respondent, with an age, a socioeconomic status, a region, attitudes, and ask it to react to the concept in natural language, like a real panelist.
- We measure that text from the outside. We compare the response to a set of anchor phrases, one per level of the scale, through semantic similarity (embeddings + cosine). The closest level, or a weighted average, yields the rating.
This separation changes everything. Variance returns, the distributions look like human ones, and the correlation with real data climbs.
Why it works
A verbatim contains a thousand nuances that a number flattens: "I'd have loved a less sweet version" isn't a "no," it's a "yes, but." Semantic similarity captures that nuance and projects it onto the scale, instead of forcing the model to make an artificial call.
It's also what makes the method defensible: it's not "a chatbot taking guesses," it's a reproducible measurement protocol, with versioned, language-validated anchors.
What Panelia does with it
At Panelia, this pipeline runs on hundreds of synthetic respondents per study. We aggregate the ratings into an intent distribution, compute a Top-2-Box and confidence intervals by bootstrap, and calibrate the whole thing on benchmark human data.
The result isn't an oracle. It's a decision-support tool, with its uncertainty on display: exactly what you need to screen concepts before committing a real fieldwork budget.
Want to see the full method, anchors included? Head to the Science page.