Most Published Research Is False

Over the years, this blog has been highly skeptical of marketing, advertising, and media research.

What passes for research in our world would be laughed out of most reputable scientific laboratories.

• We almost never use controls
• We almost never replicate our work 
• We don't have peer review 
• We don't have others see of they can reproduce our results.

There are so many ways for research to go wrong, that not bothering with any of these fundamental necessities of valid science creates enormous problems. Consequently, among sensible advertising people, there is widespread skepticism about research data and the interpretation of data.

Personally, the only thing I trust our researchers to do competently is to count. They can usually give us a pretty good idea of "how many." But asking them for a "why" or a "what" or a "how" is likely to get you an opinion masquerading as a fact.

The problem was further impressed on me recently when I read a piece by George Johnson science writer for The New York Times. Johnson writes about Dr. John P. A. Ioannidis, "a kind of meta-scientist who researches research."

Dr. Ioannidis wrote a paper in 2005 called “Why Most Published Research Findings Are False.” According to the article, "Dr. Ioannidis devised a mathematical model supporting the conclusion that most published findings are probably incorrect."

Now let's be clear.  Ioannidis is writing about real research, the kind that is done in biology and physics labs. Not the baloney that we call research.

Johnson also relates the story of the chief scientific officer of a pharmaceutical company who set about to reproduce the results of 53 "landmark papers about cancer." In 47 of the 53 cases he and his colleagues could not reproduce the results "even with the help of the original scientists working in their own labs."

Anyone who thinks cancer research is problematic but advertising research is reliable needs professional help.

By the way...
...recently a government study found that obesity among young children had plummeted 43% in the past ten years. I'd love to see Dr. Ioannidis get his hands on this baloney.