Why Copy Testing Is Bogus

"My friend shot a 158," Joe said.
“That’s the worst round of golf I’ve ever heard of,” said Mike.
“No it’s not. It’s a great round of golf,” said Joe.
“You’re crazy,” Mike said. “How can you say that?”
“He only has one arm.”

I made up that little story to make a point. The point is about copy testing, and we'll get back to it in a minute.

Copy testing is despised in the ad industry for a number of good reasons. The MBA's hate it because it's unreliable and doesn't often correlate with real-world effectiveness. The creatives hate it because it rarely rewards non-linear thinking.

I hate it for a whole other reason. Having once been a science teacher, I have a healthy regard for the difference between science and bullshit. And copy testing ain't science.

Copy tests almost always fail to include controls. Controls are the one essential of all experimentation. Research without controls is like baseball without an umpire.

A control serves two purposes:

1. To make sure you’re studying what you think you’re studying.
2. To give you a factual basis for comparison

A perfect example of providing controls in research is the placebo in medical research. When studying a new drug, scientists always give some of the subjects sugar pills. Sugar pills have no effect. If a drug they are testing seems to provide benefits, but those benefits are no greater than the benefits of taking the sugar pills, they know that the new drug is worthless. The apparent benefits are not the result of the drug, but simply of giving the subject a pill—any pill.

By testing a known quantity (the control/placebo) along with the unknown quantity (the test drug), you have a factual basis for comparison. A scientist who did a study without controls would be laughed out of any lab in the world.

Most copy tests do not use controls. They use norms. It’s important to understand that norms are not controls. Advertising norms do not tell you about ads in your category, at this time, with these people, under these circumstances.

In order to evaluate a new toothpaste campaign, for example, an advertiser needs to test it along with a campaign in the toothpaste category that is known to have done well in the real world. This known campaign should be tested with the same people, at the same time, under the same circumstances as the campaign in question. Only then can you know what success will look like.

Without that, if you get a lousy result, you don’t know why. Is the new campaign lousy? Or was the tester irritating? Or were the M & Ms stale? Or would any ad about toothpaste do poorly at this time, with these people, under these circumstances?

If the control campaign did well at the same time under the same conditions, then you know the new campaign must be lousy. But if the control campaign also tested poorly, then maybe ads about toothpaste only have one arm.


By The Way:
One reason direct marketing people think of us as idiots is that when they test they always use controls.

When Bad Things Happen to Good Campaigns

If you want to screw up a good campaign, take a terrific character and turn him into a pitch man: Exhibit 1:



The Better Blogger

TAC wrote a piece last week called The Bitter Blogger in which he railed against bloggers who did not include The Ad Contrarian on their blog rolls.

Attentive reader The Denver Egotist noticed that TAC himself does not have a blog roll. To atone for this, TAC would now like to acknowledge those wise and gentle bloggers who include TAC on their blog rolls -- Dear Jane Sample, AdPulp, thegirlRiot, Timothy Coote, Fabio, James Hipkin, DeepTechDive.

These are not the sniveling weasels I was talking about. These are delightful, charming people who deserve your attention.

Teen Commitment


Every young person in America is committed to saving the planet, unless it requires taking a bus or turning off a light.

Why Fallon Is Untrustworthy

The Wall Street Journal reports that Sony is pondering whether to give the global ad assignment for its Bravia brand of flat panel televisions to Fallon.

Fallon has done a wonderful job of making Bravia a success with great advertising (see spot below.) So what's the problem?

Well, Fallon may not be big enough to really fuck this thing up.

You see, super-size knuckleheads really like to hang with other super-size knuckleheads. And according to the Journal, Sony's afraid that Fallon's not inter-galactic enough.

Apparently, to screw something like this up, you really need to go out and find yourself an agency that's internationally stupid. When you're smart instead of huge, you just can't be trusted.



And One More Thing:
Any client foolish enough to believe the bullshit about global agencies giving them "integrated services throughout their worldwide network" needs a thorough psychiatric evaluation. Anyone who's worked at one of these worldwide monstrosities knows that not only do the offices not cooperate, they go out of their way to fuck each other. Been there, done that.