Version 1 — 18 November 2026
Esomar's 10 Questions for buyers of AI research, answered.
In July 2026 Esomar replaced the buyer-facing part of its 20 Questions with ten questions every buyer should ask a supplier of AI-based research services. We answered the twenty in public in August. Here are the ten, answered the same way: what is in place today, where you can check it, and what comes next.
Source: Esomar — 10 Questions Buyers Should Ask Suppliers of AI-Based Research Services. The questions are Esomar's; we paraphrase them and keep their order. Read the original for the points Esomar asks buyers to consider and the evidence it suggests requesting.
How to read the colours
Esomar offers buyers an optional traffic light for each answer: green, amber or red. The colours below are our own assessment of our own answers, which a buyer should not take on trust: form your own.
- GreenIn place, with evidence you can check, for the uses we recommend.
- AmberIn place in part. What is missing is named, with its next step.
The ten answers at a glance
- 1. What AI does the service use, and what for?Green
- 2. Who provides, operates and controls the AI?Green
- 3. What data does the AI use, and where does it come from?Green
- 4. How are our data, materials and outputs accessed, kept and reused — and do they improve anyone's AI?Green
- 5. What shows that the service is valid, reliable and fit for the intended use?Green
- 6. How do you handle bias, hallucination, synthetic output and poor-quality results?Green
- 7. What professional review and oversight is built in, and how are problems escalated?Green
- 8. How do you protect privacy, confidentiality, security and intellectual property, across locations?Green
- 9. How do you tell buyers about AI use, synthetic data and synthetic personas, and their limits?Green
- 10. What can you hand a buyer as evidence, audit trail and contractual commitment?Amber
1. What AI does the service use, and what for?
GreenCommercial language models write the simulated respondents' answers and draft the analysis. Who is in the sample, in what proportion, and how the results are weighted and checked is decided by code and official statistics, not by the model.
FlashInsight builds panels of simulated respondents from official population statistics, then puts a quantitative questionnaire, a one-to-one interview or a focus group to them. General-purpose language models produce each respondent's answers, in the language of the market, and draft the analysis. A model also answers the short probe questionnaire from which each sample's reliability score is computed. That is the whole of the AI in the service.
What does not depend on AI: who is in the sample and in what proportion, which comes from national statistics; the weighting of results, which follows the real population structure; the rules that turn probe answers into a reliability score; the quotation check on qualitative reports; and the AI notice on reports and teasers. These are deterministic code.
What it is for: screening and comparing options — concepts, names, messages, packs, films — before money is committed to fieldwork, and exploring the reasons behind a reaction in qualitative interviews. What it is not for: predicting what one person will do, measuring an absolute level, or replacing a tracking study. Results are read as rankings, gaps and oppositions between groups.
The service is in production, not a pilot. Studies can be run by our team or by the client, on the platform or from the AI assistant the client already uses, through a secure connector. The deliverable is a report, and the study stays open in the client's workspace for further waves or questions.
Where to check
How it works · Our method · AI usage framework
2. Who provides, operates and controls the AI?
GreenFlashInsight designs, operates and answers for the service. The language models come from several commercial model providers, chosen by us for each task and validated before use; changing model does not change the method.
FlashInsight is the research supplier and the operator. We configure the service, choose the models and their settings, and are accountable to the client for what the platform produces. The service is led by Dr Laurent Florès, who has spent more than thirty years in market research and is a past president of Esomar.
The language models are commercial models from several providers, reached through a model-routing layer. We choose the model for each task — writing a respondent's answer, conducting an interview, drafting an analysis — and validate it before it goes into production. The population, the weighting and the checks sit outside the model, so a change of model does not change the method, and we re-run our validity checks after one.
Hosting, database and payment providers are named in our privacy policy. The model layer is described there as a category rather than by supplier name, because it is a setting we change; we disclose the precise services in use to a client on request, before a study starts.
The buyer does not choose the model, the hosting region or the retention settings: we set them, and they are the same for every client. This is deliberate: it keeps one study comparable with the next.
When a client runs a study themselves, the same populations, checks and notices apply. The client decides what to test; we remain responsible for how the platform works. Access from an AI assistant goes through a standard authorisation that only an administrator of the client's workspace can grant.
Where to check
Privacy policy · Legal notice · About the author
3. What data does the AI use, and where does it come from?
GreenOfficial statistics and surveys of real people, named in full, for nine markets. We recruit no participants and collect no answers ourselves, and use client data only when the client supplies it to calibrate their own study.
Every population is built from named sources. The make-up of the population comes from official statistics: INSEE (France), ISTAT (Italy), INE (Spain), Destatis (Germany), the Office for National Statistics (United Kingdom), the US Census Bureau, INEGI (Mexico), the Haut-Commissariat au Plan (Morocco) and Eurostat, which also provides the margins for Poland, added in November 2026. Values are calibrated on surveys of real people: the European Social Survey, an academic survey conducted face to face, for the European markets; the World Values Survey for the United States and Mexico; and a national survey of 1,016 respondents in Morocco, run by a research partner.
Four kinds of data, kept apart. Public data: the statistics and academic surveys above, which anyone can download and check against what we say we reproduce — the Moroccan survey excepted, which belongs to our partner. Participant data: none of our own — we recruit nobody and collect no answers from real people; the surveys above reach us aggregated or de-identified by the bodies that ran them. Client data: only when a client supplies it to calibrate a population for their own study. Synthetic data: every answer in a study is simulated, and labelled as such.
Each market is studied in its own language, with a population built for that market; seven languages are supported today. Where a dimension is missing from a country's published statistics, we say so rather than borrow it from another country.
Freshness is a real constraint. These sources publish on multi-year cycles, so between publications our populations describe the country as last measured.
Where to check
Calibration, and what a buyer can check · European Social Survey · World Values Survey · Eurostat
4. How are our data, materials and outputs accessed, kept and reused — and do they improve anyone's AI?
GreenClient material is used for that client's study only, and never to train or tune a model, ours or anyone else's. Every model call that carries study content refuses data collection by the provider, and a client's studies are visible only to its own workspace.
What a client uploads — a concept, a film, a brief, data to calibrate a population — is used for that client's study and nothing else. It never enters a model's training, and it is never applied to another client's work. Because our corrections live in the inputs and in the checks rather than in a model's weights, we have no model to improve with client data in the first place.
Every model call that carries a study's content — answers, interviews, analyses — goes through one client that adds a setting refusing data collection by the provider, and that setting is covered by our automated tests. Whether a provider honours it is governed by its own terms, which we check but do not enforce ourselves.
Inside the platform, a client's studies are visible only to the members of its workspace. That separation is enforced in the database itself, not only in the interface.
The client owns the deliverables and the results of their study, and may use them in their own tools, AI tools included. We keep the right to use aggregate, anonymised performance data about the method — reliability scores, validation metrics — to improve and publish on the method. We publish no client content and no study result without permission. Retention periods are set out, purpose by purpose, in our privacy policy.
Where to check
Privacy policy · AI usage framework
5. What shows that the service is valid, reliable and fit for the intended use?
GreenBy measuring it against real people and publishing the results: three published measurements, two of them against real people, a dispersion correction measured market by market against surveys of real people, a reliability score on production samples, and a technical note archived so that anyone can re-run it.
Three measurements, each answering a different objection, each from a different protocol, and not to be read as a range. Stability: our simulated respondents reach 26% of the human self-consistency ceiling, the level at which real people reproduce their own answers two weeks apart. Match with reality: ρ ≈ 0.83 against Twin-2K-500, a published US benchmark of real individuals, and ρ ≈ 0.97, mean absolute error 0.35, against the European Social Survey in France. Coherence: 18 of 18 directional predictions matched established theory.
Dispersion, the defect most specific to simulated respondents, was corrected in September 2026. Simulated respondents who share a profile now disagree between 0.85 and 1.07 times as much as real respondents of that profile (0.96 to 1.07 on attitudes, 0.85 to 1.00 on personality), where they previously disagreed between 0.20 and 0.76 times as much — measured in each of the eight markets then served, against a survey of real people in that market. Poland was measured the same way before it opened.
In production, samples are given a reliability score from 0 to 100 with a traffic light, and one component acts as a gate: if contrasting profiles in the sample do not actually answer the probe questions differently, the overall score is forced to zero and the light to red. When the score cannot be computed, the screen says so rather than show one. Components, weights and thresholds are documented, and each score records the version of the method that produced it.
We do not use a model's judgement of its own work as proof of accuracy. Our checks compare outputs with real data or apply rules written in code. And our research is published so that it can be checked: our technical note on the Twin-2K-500 benchmark is archived with its scripts, prompts and outputs under a permanent identifier, and the dispersion test is described so that anyone can run it on any supplier.
Next step
Run the out-of-sample bench we have pre-registered, publish a formal measure of stability across models, and add a dispersion component to the reliability score — until then we publish no confidence interval.
Where to check
How reliable are synthetic respondents? Our own numbers · Your digital twin is more predictable than you are (archived note) · Under-dispersion, and a test anyone can run
6. How do you handle bias, hallucination, synthetic output and poor-quality results?
GreenPopulations are anchored on official statistics, a gate flags samples whose contrasting profiles do not answer differently, every quotation in a qualitative report is checked by code against the transcript, and every report and teaser says it comes from simulated respondents. We publish the failures we find, and their fixes.
Bias. Language models are known to flatten and caricature demographic groups. Our populations are drawn from official statistics rather than from the model's own idea of a group, the reliability gate flags samples whose contrasting profiles fail to differentiate, and we have published our own measurement of caricature on public opinion data.
Fabricated quotations. In a qualitative report, every quotation is checked by code against the transcript of the respondent it is attributed to. A quotation is matched ignoring only case, accents, punctuation and spacing; one found in exactly one other respondent's transcript is re-attributed to that respondent, and any other is removed; a theme left without enough real quotations is dropped, and a report left with fewer than four themes is not delivered.
Synthetic output mistaken for human evidence. There is no human data in our outputs to mix with: every answer is simulated. Every report and teaser states that its content comes from simulated respondents and was generated by an AI system. That notice is inserted by code, not written by the model, in the language of the document.
False precision. Results are read as rankings, gaps and oppositions, not as absolute levels, and we publish no confidence interval that our variance cannot support.
Corrected failures, three of them public. In August, eight out of eight injection attempts through uploaded material succeeded; after a fix, none of the behaviour-steering attempts did. In September, the compressed disagreement inside groups was corrected in every market. In November, while adding Poland, we found that the survey extract used for values counted teenagers and respondents of unknown age in the oldest age group; the calibration was restricted to adults of known age in all six markets concerned the same week.
Where to check
Why synthetic personas caricature real people (Pew data) · Under-dispersion, and what we did about it · Our 20 Questions answers, question 17 (injection tests)
7. What professional review and oversight is built in, and how are problems escalated?
GreenEvery study we deliver is read and interpreted by an experienced researcher before it goes out. Studies clients run themselves carry the same automatic checks, and there are no human participants to protect or reach.
A researcher designs the study, reads the output and writes the analysis. The model produces answers; it does not produce conclusions. Every study we deliver as an analysis is read by an experienced researcher before delivery, and today that review is hands-on, not delegated.
What the reviewer is trained to catch is the confident but wrong answer: a result that reads well and contradicts what is known about the category, a group that answers too uniformly, a quotation that sounds right and says nothing. These are matters of experience, and no score replaces them.
When a client runs a study themselves, the safeguards that do not depend on a person still apply: the reliability score and its gate, the quotation check on every qualitative report, the notice on every report and teaser. A qualitative run can be stopped while it is in progress, and our team can be reached at any stage of a study.
Participant safeguarding, which matters for AI interviewers talking to real people, does not arise here: every respondent is simulated, and no real person is interviewed.
Next step
Record the review itself — what was read, what was flagged, what was changed before delivery — so that oversight can be shown as well as stated.
Where to check
AI usage framework · Our 20 Questions answers, question 12
8. How do you protect privacy, confidentiality, security and intellectual property, across locations?
GreenThere is no participant personal data to protect: simulated profiles are drawn from aggregate statistics, not from any person's record. Client material stays the client's, isolated in its workspace and stored in Ireland; the application runs from Paris.
Participant privacy is protected by design rather than by policy: populations are drawn from published aggregate distributions, never from an individual's record, so no profile is copied from a real person. The academic files we use for validation are released in de-identified form by the bodies that collected them.
Client confidentiality and intellectual property are a separate matter, and treated separately. Client material remains the client's property, is used for that client's study only, and is isolated in its workspace at database level. Like any business, we also hold ordinary customer data — accounts, billing, access logs — governed by our privacy policy.
Location. Our application runs from Paris, and accounts, billing and stored study content are kept in Ireland. Study content sent to a model may be processed inside or outside the European Union, including in the United States, depending on the model chosen; on request we say where, and through which services, before a study starts, and a study can be set up for a client whose policy rules out processing outside the Union. Several of our providers are US companies whose own terms allow processing outside the European Union in some cases. Our privacy policy sets all of this out.
Security is tested against named risks rather than asserted. The most concrete one in our architecture is instructions hidden in uploaded material; we tested it on the live path, published the initial failure, fixed it and re-measured it.
Next step
An independent security audit and a certification: we have neither yet, and we do not claim otherwise.
Where to check
Privacy policy · Our 20 Questions answers, questions 15 to 18
9. How do you tell buyers about AI use, synthetic data and synthetic personas, and their limits?
GreenEvery report and teaser says, in its own language, that it was generated by an AI system from simulated respondents. Our method, our validation results and what the method should not be used for are published.
In the deliverable. Every report and teaser carries a notice that its content was generated by an artificial intelligence system and that the answers come from simulated respondents, not real people. The notice is inserted by code in each of the seven languages we support, so no report or teaser can go out without it.
In the commitments. Our published AI usage framework commits us never to present a simulated answer as a human one, to state what was simulated and from which sources, to publish our validation results including their limits, and to say in the deliverable what the method should not be used for.
In the words we use. We call our respondents simulated, never real, and we describe what they produce as readings of a population, not measurements of people.
In public. The method, the sources, the validation results and the open limits are on this site, dated, including the answers that have since been overtaken.
Participant-facing disclosure, which Esomar asks for when real people talk to an AI interviewer, does not apply: no real person takes part in our studies.
Next step
Extend the same notice to the raw data exports a client can download from the platform; today it covers reports and teasers.
Where to check
AI usage framework · Our method
10. What can you hand a buyer as evidence, audit trail and contractual commitment?
AmberDated public answers, published method notes, an archived and re-runnable technical note, reliability scores recorded with their method version, and contractual terms on ownership and confidentiality. No independent audit yet.
What we can hand a buyer today: this page and our 20 Questions answers, each dated, with later changes added rather than edited in; our method notes and validation results; a technical note archived with its scripts, prompts and outputs so that it can be re-run; the sources of each population; and reliability scores that record their method version, so that a change of method never silently rewrites an old score.
What we commit to: never fabricate or misrepresent research data or findings, by AI or any other means; never present a simulated answer as a human one; use client material for that client only; and date every material change to the method in public. Ownership, confidentiality and permitted use are set out in our master services agreement and in each engagement letter.
Why amber. Esomar asks for an audit trail of professional review and for independent quality checks. The review is real but not yet recorded, and no party without an interest in the result has audited the method. Our sources can be checked by anyone; our measurements are ours until someone else runs them.
Next step
Record the review trail (see question 7), keep a public record of how often our stated confidence matched the result, and invite an independent party to re-run our measurements.
Where to check
Evidence · Our 20 Questions answers, with their dated update
What has moved since our 20 Questions answers
Our answers to Esomar's 20 Questions were published on 31 August 2026 and remain online, with a dated update. Since then:
- September 2026 — the compressed disagreement inside groups was corrected in all eight markets then served, and measured against a survey of real people in each.
- October 2026 — clients can run their studies from the AI assistant they already use, through a standard authorisation granted by their workspace administrator.
- November 2026 — our technical note on the Twin-2K-500 benchmark was archived with its scripts, prompts and outputs under a permanent identifier.
- November 2026 — Poland became our ninth market, in Polish, calibrated on the same public sources and measured the same way.
- November 2026 — the values calibration was restricted to adults of known age in the six markets that use the European Social Survey.
- November 2026 — in one-to-one interviews testing several materials, the order in which they are shown now rotates between respondents.
Why we answer in public
Esomar wrote these questions for buyers, to be put to suppliers one by one. We would rather answer them once, in public, where any buyer can read the answers before the first call, and where anyone can tell us we are wrong. When we published this page, on 18 November 2026, our search found no other supplier answering these ten in public; if one does, we will link to it.
The answers will change as the service does. When they do, we will add a dated update rather than edit them in place.
Further reading
Our own public notes on the questions above. They are our work, not third-party evidence, and should be read as such.