Why Big Data Will Never Replace Market Research

According to a Digital Universe Study every two years the quantity of data is doubling and by 2020 it will reach 44 trillion gigabytes. The study also pointed out that less than 5% of data that is deemed useful is actually analysed. 

In the article below Allan explains why marketers might want to take inspiration from the data collecting giants like Facebook and Google and continue prioritising market research despite the mass amount of data they have in their pockets. Even though big data presents to us a sweet shop full of information it doesn't tell us the WHY this so. Allan explains...

Why Big Data Will Never Replace Market Research

It ain't easy being a researcher these days. Scanning the news and industry blogs, there is no shortage of articles claiming the death of market research, or how Big Data will render the research industry obsolete. According to these pundits, whereas in the past we suffered from a lack of data – and thus needed market research to fill the knowledge gaps, today we are fortunate enough to have an abundance of information. As a result, with the right Big Data techniques and talent to sift through the mountain of data, all our questions will be answered, and market research can finally take a bow and exit the stage.

Hogwash

First, let me state the requisite disclaimer. I am by no means bashing Big Data. In fact, I am very excited by the large and rich data sets that will hopefully lead to new learning and interesting insights.  My point is Big Data is no panacea. It can tell us what has happened in the past, and perhaps infer future events, but it has limited ability to explain WHY something has happened. Without understanding the WHY, Big Data is not particularly actionable.

To illustrate this point, let’s focus on the two of the biggest Big Data companies in the world.

There is no doubt that Facebook’s colossal growth has been fueled by Big Data. Every photo, like, comment, and so on that you share, is gathered, harvested, and analyzed to create a highly personalized profile of its users. With such a vast treasure trove of personal data, what possible need could Facebook have for old school market research methods?

Well, as reported by the Huffington Post, Facebook not only “regularly polls its members about their Facebook experience” but also has created a Facebook Feedback Panel to harvest the type of longitudinal research that many skeptics have already claimed is dead. Despite its massive amount of user data, clearly Facebook sees the value in directly surveying its members. 

The other Big Data behemoth is Google, which triangulates personal data from its search, email, maps, browser, and other products to create a highly precise portrait of who we are, to better target us with advertising. With an Olympus sized mountain of diverse and detailed datasets, not to mention a team of super-smart data scientists, surely Google has no need to actually ask questions of its users. The very notion of conducting such market research seems as quaint as the rotary telephone.

In fact, according to thenextweb, Google has been surveying its users to better understand WHY users react in certain ways to various advertising. Big Data can tell Google WHAT users are doing (muting an ad, closing an ad midway, etc.) but it cannot explain WHY users are reacting in that way. Thus the need for traditional market research.

Cynical pundits who like to claim Big Data will replace market research, often point out the shortcoming of various research methodologies, such as how focus groups suffer from small samples and are not truly representative. But to use that argument to strike down the entire market research industry is an example of reductio ad absurdum – a reduction to the absurd, where an argument is followed through to a ridiculous conclusion.  It is like saying that iPads are not ideal for typing long documents; therefore, no one should buy an iPad.

I understand and share the enthusiasm for Big Data, but it will always have its limits in explaining the underlying motivation behind our behavior. This is something that is clear to the most successful Big Data companies, and should be clear to researchers as well. While Big Data won’t answer all our questions, it is a terrific addition to our toolkit, and should be viewed as complementary to traditional market research methods.

By: Allan Fromen

Source: GreenBook - Charting the future of Market Research