85% of what? ACCURACY VS RELIABILITY
85 to 95% QUOTED BY VENDORS COMPARES HUMAN AVERAGE SIMULATED AVERAGE
Vendors quote 85 to 95% — on averages.
HUMANSSIMULATED ANSWER SCALE TWO CLOSE AVERAGES
A close average reassures.
HUMANSSIMULATED ANSWER SCALE
But comparing needs disagreement.
HUMANSSIMULATED ANSWER SCALE ACCURACY: THE CENTRE
Accuracy looks at the centre.
HUMANSSIMULATED ANSWER SCALE ACCURACY: THE CENTRE RELIABILITY: THE SPREAD
Reliability looks at the spread.
HUMANSSIMULATED ANSWER SCALE RELIABILITY: THE SPREAD ACCURACY: CENTRE IS CLOSE
A panel can pass one…
HUMANSSIMULATED ANSWER SCALE ACCURACY: CENTRE IS CLOSE SPREAD TOO NARROW
…and fail the other.
HUMANSSIMULATED ANSWER SCALE ACCURACY: CENTRE IS CLOSE SAME DISAGREEMENT 1.00 SIMULATEDHUMAN
At 1.00, as much disagreement as among real people.
AT WHAT LEVEL? INDIVIDUAL SEGMENT WHOLE POPULATION ON WHAT PROPERTY? AVERAGE SPREAD
Ask: at what level, on what property?
85% 85% ACCURACYthe centre RELIABILITYthe spread
A percentage alone is not enough.
EXPLORE COMPARE SORT SYNTHETIC PERSONAS STUDY WITH REAL PEOPLE FIELDWORK
When the stakes are high, confirm in the field.
FlashInsight Method over Magic. flashinsight.io
Before you trust a percentage,ask what it measures.

Space: pause · arrow keys: previous / next scene · click the bar: jump to a scene