Data vs Probability

I took my data analyst with me to Las Vegas.

We spent three days at the roulette wheel. We learned a lot of little things. We did a thorough, highly detailed analysis of what went on:

• We counted how many times red came up and how many times black came up. 
• We calculated the effect of the croupier spinning the ball left or spinning it right.
• We calculated how often red came up three times in a row. Then black.
• We established the effect of the speed of the roll on the number of times it turned up red or black.
• We computed the number of people over 40 at the table and how that affected the result
• We counted and analyzed lots of other stuff.

We lost anyway.

Meanwhile the house did no data analysis. But they knew a few big things.

• They knew there were 18 red, 18 black, and 2 green. 

So they had a 5% advantage on every roll.

Consequently, they never lost.

In marketing and advertising we have two ways of doing things. We can measure every little data point or we can see the big picture.

Small picture marketers know a lot of little things:

• Who went to what website? 
• What they searched for.
• What kind of car they drive.
• Who their dentist is.
• Where they bank
• And tons of other stuff

They create tightly focused advertising and put it in front of a select number of precisely targeted individuals.

On the other hand, big picture marketers know a few big things:

• People are more likely to buy brands they're familiar with. 
• People are more likely to buy brands that sound interesting.
• People are more likely to buy brands they like. 

They work very hard to produce widely appealing materials and put them everywhere.

Then they stand back and let probability do the work.

Why have all the world's leading brands been built by big picture marketers?

Because the more you study data, the more you realize that data is just the residue of probability.

Evaluating All The Campaigns In The World

The ad industry has never been short on hyperbole and unreliability. It has been a problem for decades.

The era of online advertising has only deepened this problem.

One of the troubling new phenomena is the way third parties -- the trade press, the business press, academia, and trade organizations -- have become complicit in the reckless boosterism of online advertising at the expense of the critical thinking and skepticism that used to be considered their responsibility.

This was reinforced last week when I read an article on MediaPost entitled:  

"WARC: Over Half Of The World's Most Impactful Campaigns Are Digital-Led."

WARC is an acronym for an organization that modestly calls itself the "World Advertising Research Center."

The thrust of the article was this:

"The World Advertising Research Center (Warc) has issued its annual rankings of the most effective marketing campaigns...The Warc analysis found that 55% of the world’s most impactful campaigns are digital-led."

Upon reading this, my first thought was this: How do you evaluate all the marketing campaigns in the world? There must be tens of thousands of marketing campaigns launched every year. You'd have to have thousands of people on staff, speaking dozens of languages, monitoring all media, spending months on analyzing sales data and consumer sentiment. What a remarkable undertaking!

My next thought was, hold on, this must be a click-bait headline from a publisher intentionally misrepresenting the WARC story to make a few extra bucks. But then I went to the WARC website and there it was plain as day...

 "The Warc 100 is a ranking of the world’s best marketing campaigns..."

I was hooked. What an amazing project. I couldn't wait to discover how they determined this. Until I read the rest of the sentence...

"...based on performance in effectiveness and strategy competitions."

Wait a minute... "based on performance in effectiveness and strategy competitions?" What the hell does that mean? So I dug further.

This led me to a page that described what MediaPost called WARC's "rigorous methodology developed in consultation with King's College London." Now I'm really impressed. So I read on.

What I found was a truly majestic exercise in making the obvious incomprehensible. I invite you to read the methodology here, but I warn you -- don't do it near sharp objects.

Once you fight your way through the hogwash and the labyrinthine description, you suddenly realize that the WARC 100 is essentially an award for winning awards.

WARC, of course, wants to blind us with science, so the way they put it is...

"Warc tracks different advertising competitions around the world – all of them require entrants to show the impact of a campaign. Campaigns (and the brands and agencies behind them) are awarded points based on the prizes they win in those competitions."

Which, of course, is thinly disguised double-talk for it's an award for winning awards.

In order to swallow WARC's assertion that these are the world's best campaigns, you have to accept all of the following:

• Advertising awards are a reliable proxy for effectiveness
• WARC's choice of which award shows to consider (which they refuse to divulge) are a valid sample
• WARC's arbitrary weighting scheme is scientifically defensible.

If any one of these assumptions is erroneous, the whole formulation falls apart. It is my opinion that every one of the premises is erroneous.

Furthermore, a campaign cannot even be considered unless someone entered it in one of the awards shows WARC draws on. That alone is enough to make its claim of identifying the world's most effective campaigns spurious.

Anyone who has spent 15 minutes in an agency knows the following:

• Some agencies spend hundreds of thousand of dollars entering awards shows every year (sometimes called "award-whores".) Some spend nothing.
• Some agencies have people on staff who are dedicated specialists in generating "case histories" that are effective at winning awards.
• Some agencies have people who specialize in knowing the most favorable categories to enter to produce an award.

Further, anyone who has ever judged an award show knows the following:

• On every judging panel in every awards show there are brilliant people and complete fools. It is not unheard of for the fools to prevail.
• The judging itself is like every other aspect of life -- frequently ruled by emotion, not rationality.

Unlike many in advertising, I am not an awards hater. I think great stuff deserves to be recognized. Advertising and marketing people do a lot of suffering and are entitled to some satisfaction once in a while. If an award does that, I'm happy for them.

When I was a copywriter and a creative director I was always proud and gratified when we won a major award. But I never deluded myself into the silly belief that an award was in any way an accurate reflection of effectiveness. I knew how the game was played.

The WARC 100 winners can arguably be billed as "the world's most awarded marketing campaigns." But to bill them as "the world's most effective campaigns" is simply brazen puffery. And to draw a conclusion from this that the best campaigns are "digital-led" is fatuous and irresponsible.

The effectiveness of advertising is a serious business that deserves serious scrutiny. If you are going to make an extraordinary claim -- that you have uncovered the world's most impactful campaigns -- you have an obligation to demonstrate that you have devised a judicious, reliable methodology for determining this..

WARC's methodology is superficial and unconvincing.

If Agency People Were Honest

Here are two memos that ought to be circulated at most big agencies:

MEMO

TO:               Staff
FROM:         Human Resources
SUBJECT:   We Are Not A Family

Contrary to what you heard from senior staff here at the agency when you were hired and at several staff meetings since then, we are not a family.

Families are forgiving. We are not.

Families are tied together for life. We are not.

Families can't get rid of problem people. They can't replace them with more pleasant or capable or younger people. We can.

Families have to feed you. We don't.

Just so we're clear, here are some things you should know:

• We don't love you. Many of you we don't even like.
• When you become bad casting we will hire someone younger and prettier.
• You will have a job here as long as you produce more than you consume and solve more problems than you create.
• You will have a job here until we find someone who can do your job as well but costs less.
• You will have a job here as long as the client thinks you're necessary.

Nothing personal. We just thought you'd appreciate knowing the truth.

Sincerely,

HR


MEMO

TO:               Human Resources
FROM:         Staff
SUBJECT:    Leaving At First Opportunity

Contrary to what we said when we took these jobs, we are not really that thrilled to be working here.

As far as we can tell, this place is pretty much identical to every other place we've worked at.

And all that stuff about being a family? Please...we're your employees, not your children.

Here are a few things you should know:

We believe we are underpaid. 
We believe our bosses are, for the most part, schmucks.
We think you have no balls and suck-up to our clients.

Just so we're clear, here are a few things you should know:

• We don't respect most of you.
• We will stay here until someone is willing to pay us more.
• We will stay here until someone offers us a better job. 

Nothing personal. We just thought you'd appreciate knowing the truth.

Sincerely,

Your employees