You And Me And Monkeys Throwing Darts

Here at Ad Contrarian World Headquarters, there are a few principles that guide us.

First, we never make predictions. The reason for this is that people who make
predictions are almost always wrong. It's way more entertaining to make fun of other people's stupid predictions than to have them make fun of yours.

Another is that we never trust experts. Again, their pontifications almost always turn out
wrong and there's nothing we like better than sticking it to them. We try to base our
opinions on facts, rather than ideology. This is a point of view that is terribly out of fashion in contemporary marketing.

Just to clarify something here, there are certain types of experts we have vastly more confidence in than others.

Physical scientists, while sometimes mistaken, have a system of checks that self-corrects. They are constantly peer-testing each other's assertions, which in the fullness of time usually leads to the uncovering of fallacious ideas. Social scientists, on the other hand, are completely unreliable. They use the language and trappings of science, but their studies usually produce opinions, not facts.

Consequently, I believe experts in the physical sciences really are experts, while "experts" in the social sciences are frequently just people with credentials and loud beliefs.

Of all the social sciences, economics is the one I have least confidence in. As George Bernard
Shaw once famously said, ""If all the economists were laid end to end, they'd never reach a
conclusion."

President Harry Truman, tired of hearing economists say, "On one hand...but on the other
hand..." asked if anyone knew where he could find a one-handed economist.

Of all the disciplines within economics, marketing is the one I have toiled in, am most familiar with and, sadly, have little confidence in. Within marketing, the branches I have gone out of my way to criticize are branding, web advertising, and social media marketing.

While I believe that these fields are laden with bozos and charlatans, the truth is I have no comprehensive scientific evidence to prove it. All I have are anecdotes, and anecdotes are fun but unreliable.

This does not mean, however, that there isn't evidence in other fields that demonstrates the unreliability of many social science "experts."

Enter Philip Tetlock.

Tetlock is an author, professor of organizational behavior at the Haas Business School at the University of California-Berkeley, and the winner of lots of impressive awards.

According to CNN, Tetlock is the world's foremost expert on experts. For over 25 years, he has been conducting an experiment to quantify the forecasting skill of political experts.

Tetlock has studied 300 academics, economists, policymakers and journalists and compared more than 82,000 of their forecasts to what actually happened in the real world. Here are his conclusions:

• "We found that our experts' predictions barely beat random guesses - the statistical equivalent of a dart-throwing chimp..."

• "Ironically, the more famous the expert, the less accurate his or her predictions tended to be."

• "(Experts) go wrong when they leap to judgment or are too slow to change their minds in the face of contrary evidence."

• “Partisans across the opinion spectrum are vulnerable to occasional bouts of ideologically induced insanity.”

• "The most important factor was not how much education or experience the experts had but how they thought. You know the famous line that [philosopher] Isaiah Berlin borrowed from a Greek poet, "The fox knows many things, but the hedgehog knows one big thing"? The better forecasters were like Berlin's foxes: self-critical, eclectic thinkers who were willing to update their beliefs when faced with contrary evidence, were doubtful of grand schemes and were rather modest about their predictive ability. The less successful forecasters were like hedgehogs: They tended to have one big, beautiful idea that they loved to stretch, sometimes to the breaking point. They tended to be articulate and very persuasive as to why their idea explained everything. The media often love hedgehogs."

From a New Yorker article about Tetlock's book Expert Political Judgment:

• "When (experts are) wrong, they’re rarely held accountable, and they rarely admit it, either. They insist that they were just off on timing, or blindsided by an improbable event, or almost right, or wrong for the right reasons."

• "The accuracy of an expert’s predictions actually has an inverse relationship to his or her self-confidence, renown, and, beyond a certain point, depth of knowledge."

• "Expert Political Judgment is just one of more than a hundred studies that have pitted experts against statistical or actuarial formulas, and in almost all of those studies the people either do no better than the formulas or do worse."

Listening to Tetlock speak recently, I found one of his points particularly gratifying. He said that an algorithm that predicted no change did better than the experts in forecasting the future. This coincides nicely with a little aphorism that appears every day in the right column near the bottom of this page, "Nobody ever got famous predicting that things would stay pretty much the same."

In advertising, we have always had a great many articulate, self-assured, and highly-esteemed people who pass themselves off as experts and futurists. As the art and technology of marketing have become more esoteric and arcane, and as data, metrics, and analytics have soared to the forefront of our attention, these people have achieved greater and greater status. Their pretensions are, once again, undercut by Tetlock.

"...mathematics has a certain mystique. People get intimidated by it, and no one challenge(s) the models."

For the most part, our branding, web, and social media experts (and their acolytes) tend to be enthusiasts and ideologues. This makes them unreliable. Regardless of their credentials and their alarming self-regard, they are no better at forecasting trends or results than you or me or monkeys throwing darts.

Follow them at your peril.