January 04, 2021

The Inescapable Logic of Ad Fraud

The recent alarming revelations of Russian hacking of 250 US Government agencies, which went undetected by our most sophisticated cybersecurity defenses including the military’s Cyber Command, the National Security Agency, and the Department of Homeland Security must lead us in the marketing business to reevaluate our thinking about ad fraud.

The scope of online ad fraud has been argued about for years by computer scientists, software engineers, cybersecurity analysts, advertising media specialists, and independent researchers.

On one side we have advertising and marketing trade organizations, agencies, and their security consultants who tell us that ad fraud is a minor problem that is being well-defended and, in fact, is shrinking annually

On the other side we have independent researchers who tell us that ad fraud is a massive problem (recently estimated at over $60 billion) that is becoming harder to identify and is growing dangerously.

Both sides provide metrics and data that purport to prove their point. Who should we believe? 

I would like to argue this proposition from a new point of view -- from the point of view of those of us who are not computer scientists and cannot interpret the impenetrable computer code that underlies cyber theft, and with the added knowledge of the recent shocking revelations about undetected hacking.

Rather than a mathematical or data driven argument, I will present a theoretical argument. Instead of data, I will provide logic. 

Let's start with indisputable facts:

   - The online advertising marketplace trades over $300 billion annually via computer systems.

   - Hackers - in particular state sponsored hackers - have recently been shown to have the ability to penetrate some of the most "secure" systems in the world, undetected.

   - Every person, business, or government agency that has ever been hacked had authoritative assurances that it was secure -- until it turned out it wasn't.

   - There are a multitude of ways that criminal actors have discovered for extracting money from the adtech ecosystem.

   - Gaming the programmatic ecosystem (which transacts about 80% of online ad activity) has been shown to be astoundingly simple.

   - There is no international governing authority, and consequently there are no cross-border penalties, for committing online ad fraud.

Now some assertions on my part:

It is folly to believe that hackers who can penetrate systems protected by the US military’s Cyber Command, the National Security Agency, and the Department of Homeland Security without detection could not easily penetrate adtech systems without detection.

There are governments in the world with both very sophisticated technology operations and economies that would massively benefit from the addition of billions of dollars.

Now some logic:

If the Cyber Command, the NSA, and the Department of Homeland Security can be fooled, I don't think it's a stretch to assume that fraud detection software can also be fooled. Consequently, if state sponsored hackers are fiddling the adtech ecosystem, it's likely that ad fraud detection systems aren't seeing it.

It would be amazing if state sponsored cyber criminals didn't view the adtech marketplace as ridiculously easy pickings and even more delicious since there are no consequences for being discovered.

Some conclusions:

If state sponsored penetration of adtech systems exist, the commercial fraud detection companies should be considered seriously overmatched. And, of course, the bold assertions of trade organizations, agencies or marketers are no more reliable than those of the fraud detection companies they rely on.

While we know that criminals and criminal organizations are active in stealing money from the adtech systems, we don't know if governments are. In light of recent revelations, however, it seems highly likely that state sponsored cyber operations would be powerfully attracted to the tens of billions of dollars that the adtech ecosystem is unwittingly dangling in front of them. If so, ad fraud is probably a lot harder to detect and a lot larger than anyone thinks it is.

Let's boil this down to two simple questions...

If you were a bad guy, and you could easily steal billions of dollars with a tiny possibility of detection and no possibility of consequences even if you were detected, why wouldn't you?

If you are a marketer spending substantially on digital advertising, what reason do you have for believing the metrics you're getting?

December 21, 2020

Part 2: Is Creative Advertising Really More Effective?

Last week I wrote a post that posed the question, "Is Creative Advertising Really More Effective?" 

As someone who has been a lifelong advocate for the power of creativity in advertising, I admitted that while I believe the answer is a resounding "yes," I don't know of any rigorous studies that could prove it to a scrupulously scientific skeptic.

The post elicited a healthy conversation on Twitter and LinkedIn. Here is a recap of some of the comments, and my reaction to them.

- Several people pointed me to studies that they believe prove the case. The ironic thing is that these are studies I myself have used to argue in favor of the proposition. But in trying to be intellectually honest with myself, while I personally believe the findings of the studies, I see imperfections in the methodologies that, in my opinion, would disqualify these studies as rigorous science to a meticulous researcher.

Principal among the imperfections is the assessment of creativity. In order to get a scientifically valid understanding of the effect of creativity on effectiveness we need to start with a pure assessment of creativity. Awards or other forms of industry recognition do not meet my standard of scientific validity. Here's why. Let's assume, as the awards shows do, for the sake of argument that it is possible for experts to competently assign assessments of creativity. In many, if not most cases, the people involved in evaluating creativity may have been exposed to the advertising and, directly or indirectly, to commentary about the advertising for a year or more. They also may have knowledge of the agencies or individuals who are responsible for the advertising and the creative reputations of these agencies or individuals. If this is the case, their evaluation of creativity may have been contaminated by cultural expectations or knowledge of, or inferences about, the effectiveness of the advertising.

If we are to be rigorous in our assessment of creativity our methodology needs to adhere to the accepted standards for all other types of rigorous research.  In which case the experts assigned to assessing creativity should be required to do so blind. They should do so without knowing the following:

- Who created the advertising
- Any commentary on the advertising
- Any knowledge of the success or failure of the brand in question

This is the only way we can get a pure assessment of creativity without the unconscious contamination of outside influences or a priori inferences of success by the judges.

Once we have a rigorous, uncontaminated assessment of creativity, we can compare that to business results and get an unambiguous answer to our question (at least in my opinion.)

   - Several people commented that the only criterion for creativity in advertising is sales success. I reject this out of hand. Without getting into a deep philosophical discussion, let me give three simple reasons why this is not acceptable to me. 

First, I would point to the argument made by Byron Sharp in "How Brands Grow" that one of advertising's primary functions is not necessarily to grow sales, but to maintain sales and market share. Or as he says, keep the airplane at 35,000 feet. In a highly competitive world, it can take an effective advertising effort just to keep many high-flying brands aloft. This is rarely taken into account in most analyses of ad effectiveness.

Second, I would argue that the long-term effect of advertising on brand success is very hard to tease out of sales results that are calculated on shorter time scales.

Sales effectiveness over the course of the time periods taken into account by awards shows is not necessarily indicative of the big picture effectiveness of the advertising in question. I will once again defer to Prof. Sharp as well as Mark Ritson and Binet and Field who all make a compelling case for assessing the effectiveness of advertising over years (by the way, if you want to hear a so-called marketing expert who seems to be completely ignorant of these effects, listen to this idiot explain why TV advertising doesn't work.)

Third, one of the things that makes advertising a fascinating subject (and a frustrating one to practitioners) is the role of probability. While I firmly believe that creativity in advertising is a massive advantage over banality, I also recognize that advertising I deem highly creative has an inconvenient record of failure. In advertising there are no alwayses or nevers, only likelihoods and probabilities. I think I can safely predict that when the day comes that I am satisfied I have seen a scientifically valid description of the relationship between creativity and effectiveness, creativity will be found to be not a guarantee of advertising success, just a more likely outcome.

Furthermore, and perhaps most important of all, if you assert that the only criterion for creativity is effectiveness, then you are trapped in a tautology: Creative advertising is more effective because effective advertising is, by definition, more creative.

   - Inevitably, there were the dreary semantic arguments. What do we mean by "creative?" What do we mean by "effective." I don't want to go down that rabbit hole because there is no way out. Let me just assert (without an ounce of proof) that competent ad people know what we mean by "creative" and competent business people know what we mean by "effective." Let's leave it at that.

Just as in any form of art or craft, creativity is often experienced subjectively. But that doesn't mean it has no objective reality. To define creativity strictly as a function of sales success is to reject creativity as an objective reality. To do so in advertising is no different from repudiating it in all forms of art, music, and literature. Advertising may not have the same goals or gravitas as art, music, or literature, but it can still be measured by the same standards of excellence. It also can be subject to the same pitfalls. Creativity without purpose can soon become indistinguishable from self-indulgence.

Let me repeat what I said last week. I am not a scholar on the subject of advertising research and I am not aware of all the literature on it. Maybe there exists a study I am not aware of that proves the case and would meet my personal standard of scientific rigor. In fact, I hope that somewhere there is.  

Until then I will be stuck in my personal predicament. Do I believe creative advertising is really more effective than mundane advertising? Without question. Can I prove it to you? Not exactly.

December 17, 2020

Is "Creative" Advertising Really More Effective?

As long as I've been in the advertising business there has been a very large question smoldering under the surface of my skin: Does advertising that we deem to be more creative actually produce better business results, or is that just a fond wish that "creatives" and our supporters have invented to justify treating advertising as an art, and not just a blunt instrument?

As a former copywriter and creative director I am a strong believer in the power of creativity in advertising. In fact, every neuron in my tiny little brain is committed to this belief. 

But there is another part of my brain (the part that used to teach science) that tries to remind me about intellectual honesty, and keeps saying to me, "How do you know this?" 

I am not a scholar on this subject. I have not gone through all the literature and all the studies. But I have been exposed to some of the research on the subject and it worries me. 

The studies that I have seen and read generally seem to take the following form. The researcher starts with a group of ads that have been recognized as exceptionally creative by experts or by respected awards organizations and compares their real-world business effectiveness to advertising that has not been recognized as such. The results are often convincing, and the "creative" ads exhibit significantly superior effectiveness.

An argument one could make against this methodology (which I will not make) is that it is dependent on two factors that ought not be taken at face value. First, that the experts and award committees are actually able to accurately discern levels of creativity. Creativity is a notoriously difficult thing to define and the idea that the people who have been tasked with defining it are particularly qualified to do so is a difficult case to prove. 

The second argument against this methodology is about the business results that are used to measure effectiveness. How do we know they are reliable? As someone who has written more than his share of case histories, I am very sensitive to the effect that imaginative writing can play in the description of success.

If the people assessing creativity are not uniquely qualified to do so, and if the measures of effectiveness are not wholly reliable, then the conclusions cannot be taken seriously.

But I am not going to criticize the methodology on this basis. For the sake of argument, let's assume that the experts and awards committees are fully qualified to define and assess creativity and the metrics that are used to define business success are fully accurate.

I still have a problem.

Creative awards are usually presented in the year following the initiation of a campaign. You can't give awards for advertising created in 2020 until the year is over. Consequently, awards committees and experts usually don't get together to make their determinations until "awards season" a few months into the following year.

So there can be a lag time of between 12 and 18 months between the time a campaign launches and the determination of its level of "creativity" by the experts. In this lag period there is every opportunity for the people who are going to be charged with determining creativity at a later date to be exposed to business results of campaigns. Trade publications, advertising insiders, the business section of newspapers, and industry gossip are reporting on winners and losers every day of the year.

It is highly likely that the experts are reading and hearing reports of advertising successes and failures throughout the year. By the time they are tasked with determining levels of creativity, the experts and the awards committees have a very good idea of what campaigns produced highly effective advertising the previous year and what campaigns fell flat. Is it realistic to expect these people to ignore what they know about success and failure when they are assessing levels of creativity?

I find that hard to believe. It seems to me only natural that an individual will give higher grades for creativity to a campaign she knows to have been effective than to one she knows to have bombed. It seems highly unlikely that an awards judge will deem a campaign very creative if he knows the campaign was a disaster, the agency was fired, the marketing director replaced and the campaign pulled off the air. 

I am not implying that experts and awards committees are remiss in their duties or unprincipled in their decision making. I am merely suggesting that they are human. The likelihood that a human will take something he knows to have been a massive failure and compare it favorably to something he knows to have been a massive success is not high.

If this is the case, then the process can be, to a worrying degree, a tautology. Campaigns known to have been effective are presented as being highly creative, and campaigns thusly deemed highly creative are presented as proof of superior effectiveness.

It can be a very simple but obscure example of circular logic.

I still firmly believe that creativity is the single most important determination of advertising effectiveness. But I wish I had a more substantial, scientific basis for that belief.

See Part 2 of this piece here.