Junk Research, Shabby Journalism, And Social Media

Recently, a colleague sent me an article from Fast Company called What Women Want: Facebook Ads!
 
The article was about the amazing ability of Facebook to influence female consumers.

"Businesses may have once had to guess how their ads are being received, but no longer..."

...is how Fast Company put it.

The article ran in January of this year and at first glance offers a pretty impressive array of statistics about the power of marketing to women by using Facebook.

On second and third glance, however, it is a very good example of boosterism disguised as journalism, and self-serving manipulation masquerading as research.

Here are some of the quotes from the Fast Company article:

"Half of the respondents said they bought a product in 2009 because of something they'd seen on a social networking site."

"...a whopping 80% of the women polled said they had (become fans of brands or products)"

"...it is excellent data for retailers searching for clues about competing in this pinched consumer world."

Well, this is pretty convincing stuff. And there was more. So I decided to explore a little further.

As the source of its information, the Fast Company article linked to an article called Women Warm Up to Brands on Social Sites on a website called eMarketer Digital Intelligence.  From this article I learned that:

"80% of female Internet users said they had become a fan of a product or brand on a social network."

"One-half of female Internet users had brought (sic) a product because of a social network."

Being a semi-open-minded kind of guy, I started thinking that maybe I needed to revise my opinion about the effectiveness of social media marketing.

But something was bothering me. Neither of the articles was very clear about the design or analysis of the research. Who designed it? Who interpreted the results? Who were the women they studied? How were they selected?

In the case of Fast Company, the respondents were characterized as "the women polled" or the "respondents." In eMarketer, the respondents were identified as "Internet users."

The article at eMarketer attributed the research to SheSpeaks “Annual Social Media Study.”

So I went to SheSpeaks to find the study. What I found was startling.

Here's what SheSpeaks says about itself.

"Our community engagement programs help brands start conversations with target consumers. We introduce your brand to consumers, create and sustain conversations, capture honest and authentic feedback, and identify and nurture brand advocates.

We help your brand story become part of her conversations."

In other words, SheSpeaks seems to make its living by finding women who are actively engaged in online social activities and signing them up. Then they further encourage these women to use social media tools to advocate for their clients' brands. 

And the only women who took part in this "research" were members of SheSpeaks!

If you intentionally set out to create a sample skewed to produce a certain result, you couldn't have done better.

There is not a reputable researcher in the world who would use a sample like this to be representative of anything.

How can you possibly study this group -- and only this group -- and not get results that are hopelessly skewed? The answer is, you can't. It's like doing research on milk consumption by polling dairy farmers.

And yet, in all the reporting I read, and in the study itself, nowhere is it made clear that this group is way outside the profile of average women and the results should not be considered indicative of average behavior. In fact, quite the opposite is implied.

There is a certain aspect of the social media marketing culture that gives it an aroma of sneaky unreliability. It's like listening to a 16-year old explain how the car got scratched. This is the type of stuff that does it.

The shame is that gullible clients and naive "digital strategists" are being sold this baloney every day. 

As An Example Of How These "Facts" Spread... 
...here are a few excerpts of misleading reports about this survey from online "news" outlets:

According to Biz Report..."SheSpeaks' latest Social Media Study has been released and it shows that almost 90% of women are using popular social networks. Of interest to marketers will be their figures showing how social media is becoming a strong driver of purchase behavior." 

"A majority of U.S. women use social networking sites, and half of them say social networking sites influence their shopping habits. These are some of the key findings from a new study on online socials networking among women performed by social media platform SheSpeaks," says the website Marketing Vox.

"...Facebook, MySpace and Twitter, have emerged as key drivers of purchase intent among women, with one-half (50%) of social media users reporting they have purchased products because of information on social networking sites." says United Business Media's PR Newswire.

There are dozens more, but I'm getting hungry and you get the picture.