Launching soon on the Shopify App Store
The method
Synthetic research gets a lot of eye-rolls, and often earns them. So here is how ours actually works, where the genuinely hard problems are, and the line we draw between what we will answer and what we won't. Read it and decide for yourself.
Why this is hard
Prompted well, a language model can reproduce the rough shape of what a population thinks. That is the promise the whole category is chasing. The naive version, though, fails in ways that are now well documented: ask a model to score something and it agrees too readily, avoids the extremes, and collapses a varied population into one agreeable voice.
The output looks like data and behaves like a guess, which is worse than having nothing, because you will trust it. Everything below is how we handle each of those failures, and where we decided the right answer was to not answer at all.
The hard problems
These are the documented weaknesses of using language models as respondents. We do not wave them away. We built around the ones we can and refuse the ones we can't.
Models are trained to be agreeable, so a leading question gets you the answer you were hoping for. The field calls it sycophancy.
What we do Simulated shoppers never see a scale or a yes/no. Every answer is open text in their own words, so there is nothing to agree with.
Ask a model to rate something 1 to 5 and it clusters on 3, almost never choosing 1 or 5. A naive survey returns tidy, unrealistic numbers.
What we do We do not ask for numbers. Scores come from the language afterwards, so the real shape of opinion survives.
Left alone, models drift toward one bland majority voice and lose the tails, which makes a synthetic sample look calmer than the real thing.
What we do Audiences come from census data, not invention, and every result shows its consensus and range, never just a mean.
A model's default view is misaligned with plenty of real groups, something measured directly against live survey data.
What we do We make no claims at the segment level. Our own testing shows those cuts are unreliable, and we say so.
Change a few words in a persona prompt and the answer moves. Personas offer the feel of rigor without the substance.
What we do We do not sell personas. Simulated shoppers sit on real population structure and run through one fixed, versioned protocol.
A model cannot taste a drink, has not lived with your brand, and has no real memory of last year. Push it there and it invents.
What we do Those studies are blocked before they run. If we cannot answer it honestly, you cannot run it.
How words become a number
Simulated shoppers answer in words. To turn those words into a score we use Semantic Similarity Rating, a method published in 2025 by Maier and colleagues. Each answer is matched against reference statements for every rating level, and the score is read from where it lands. The model is never asked for a number.
We built on published, independent research on purpose, rather than a scoring trick of our own. On its authors' benchmark of 57 product surveys, the method recovered about 0% of human test-retest reliability. Grounding it in census data, gating it, and turning it into a verdict is our own work.
01 The answer
“Looks less sugary than the big brands. I'd try a box if the price was close to what I pay now.”
Simulated shopper №072 · open text
02 Matched to anchors
Similarity to each level's reference statement. No scale is shown.
03 A distribution
A probability across the scale, not a single guess.
04 Score & verdict
Combined across 100 shoppers, with a confidence range.
Where the shoppers come from
Real population structure
Audiences are built from government census microdata, roughly 100,000 US profiles and a tighter Australian set, carrying real age, location, income and occupation.
Shaped by your store
Your Shopify patterns, from repeat purchase to spend bands to product affinities, shape that base as aggregate context. They shape the audience. They never replace it.
No personal data, structurally
Customers are reduced to anonymized summaries in code, with names, emails and addresses stripped and small groups suppressed. There is no personal data to leak because none is kept.
Where it works, and where it doesn't
Language models are strongest where they have absorbed the most: everyday retail, food, drink, packaging, household goods. The method we build on says the same, that it only holds where the model already knows the domain well. Our strength in consumer is not luck. It is the shape of the tool, confirmed across a lot of categories we tried and chose not to keep.
Where we're confident
Where we're not
The label on everything
Every number we return is a signal for a decision, not a forecast of the market. We would rather be usefully right about the direction than falsely precise about the destination, and we label it that way every time, so you always know exactly what you are holding.
Don't take our word for it
The method we build on and the criticisms we design around are all public. Here is the short list: the case for this working, and the case against.
Book a demo and we'll run your first study with you.
Direct install from the Shopify App Store arrives in a few weeks.