Copy testing

Copy testing is market research that shows draft advertising copy to a sample of the target audience before it runs. It measures whether the copy is noticed, understood, believed, liked and attributed to the right brand. The aim is simple: drop the weakest versions before media money is spent on them, and learn why the best one works.

The copy can be a headline, a claim, a script, a print or social execution, or a finished film. The logic is the same. When the respondents are synthetic, the method changes less than people expect, and what the result can prove changes more.

The essentials

• Copy testing compares versions. Its most useful output is a ranking and the reasons behind it, not an absolute score.
• Synthetic respondents can rank versions early and cheaply, provided they disagree with each other as much as real people do.
• The specific risk is under-dispersion: when simulated answers are too similar, the gap between versions shrinks or swings, and the winner can change from one run to the next.
• A norm-based score and an in-market measure still need real people.

What does copy testing measure?

The battery has barely changed in decades. Attention: is the copy noticed? Comprehension: is the main message the one intended? Credibility: is the promise believed? Likeability and emotion. Persuasion or purchase intent. And brand attribution: will people remember whose ad it was? Open-ended questions add the reason behind each score, and they are usually where the fix for a weak version is found.

A group of US advertising agencies set out the principles of good copy testing in 1982, known as PACT. Several still hold: test against the objective the copy was written for, use more than one measure, compare versions on the same basis, and check that the test is reliable. The review by Pechmann and Andrews is a clear account of the methods.

What are the main copy testing methods?

• Monadic survey. Each respondent sees one version and rates it. It is the cleanest comparison, and the most expensive, because every version needs its own sample.
• Sequential monadic survey. Each respondent sees several versions in rotated order. It is cheaper, with some order effects.
• Qualitative testing. Interviews or groups that explain why a version works or fails. They are good for fixing copy, and weak for choosing between close versions.
• Live A/B testing. Versions run on real traffic and are compared on clicks or conversions. It measures behaviour, but only after launch, and it does not say why.

Copy testing, ad testing, ad pre-testing: what is the difference?

The three overlap, and the words are often used for one another. Copy testing focuses on the words: headlines, claims, scripts and body copy. It can run before any visual exists, which is why it comes first. Ad testing evaluates the whole execution, pictures and sound included. An ad pre-test is ad testing done before launch, as opposed to tracking an ad once it runs. In practice, teams test the copy early, then pre-test the finished ad. The measures are the same; only the stimulus changes.

Can copy testing be done with synthetic respondents?

Yes, for the part of the job that is comparison. A synthetic survey can run each version as a monadic cell on the same simulated audience, in hours, as many times as needed. That changes how copy testing is used: instead of testing two finished executions once, a team can screen ten headlines, keep three, rewrite them, and test again before anything goes to the field.

What reads: the gap between versions, where comprehension breaks, the objections in the open-ended answers, which version wins and by how much. What does not read: an absolute likeability score compared with a market norm, recall after real exposure, or a forecast of sales. The same split applies to the ad pre-test and to the concept test.

Why does under-dispersion matter so much in copy testing?

Because copy testing is read on gaps, and gaps depend on spread. If simulated respondents who share a profile agree with each other far more than real people do, two things happen. The averages can still look right, so the problem is invisible on the chart. But the difference between versions becomes fragile: it shrinks, or it swings from one run to the next, and a different winner can come out of the same test.

We measured this defect in our own system against national surveys in eight markets. Before correction, our simulated respondents showed 0.20 to 0.76 of the human spread. After correction, 0.96 to 1.07 on the attitudinal layer and 0.85 to 1.00 on the personality layer, where 1.00 means human-equivalent. The full figures, market by market, are in How reliable are synthetic respondents?

What should you check before trusting a synthetic copy test?

• The spread of answers compared with a human reference for the same market, named by source and wave.
• The gap between versions on a second, independent run. If the winner changes, the result is noise.
• Open-ended answers that contain objections. No execution has only strengths; uniformly warm verbatims are a warning sign.
• An audience built for the market where the copy will run, not a generic one.

The method for the first check is public: see a test anyone can run.