Data scientists and market researchers often seem to come from different planets, with data science appearing to spin fast, dark and mysterious, and market researchers feeling recently demoted and a bit forgotten. They often have different skills, with data scientists focusing on quantitative analysis of unstructured big data and market researchers on crafting more deductive stories from a combination of qualitative and quantitative insights. In leading organizations such as Amazon though, these skills go hand in hand to produce extraordinary customer experience and profitable growth. A key part of such success is the integration of data science with decision making, instead of creating separate departments to ‘crunch the numbers’ versus making business decisions. Last week, Holly Heline Jarrell discussed the match between market research and data science in the DATA Initiative at Northeastern speaker series.
‘Data doesn’t tell stories, people tell stories’ is Holly’s starting point. Many companies have data assets they don’t quite know what to do with. How do we leverage this data into actionable insights? For which decision makers does our data add most value? To address these $64M questions, we need to ask about the WHY behind the data, whether it is showing a new trend or an unexpected correlation. ‘Insight’ is the understanding of a specific causal effect in a specific context, and generates from combining information that previously was not connected. Holly’s example is a ‘discipline deficit’ among Americans observed in surveys, ethnography and cultural microtrends: In a world where many Americans feel the of control over their spending, eating, drinking and even their kids, some increasingly feel the need for structure and control. How does that related to discipline-enabling products and services companies could develop and commercialize?
In a case study, Holly described how her team integrated existing segmentation profiles with a much broader – and nationally representative – sample of respondents whose digital actions could be captured, and who could be asked follow-up questions. After satisfying the client’s initial question about how best to target a new product to specific segments, the team then set out to automatize the full system of asking business questions and obtaining reliable answers, while respecting the different privacy concerns of the different datasets. As a result, market researchers could quickly and accurately answer questions such as ‘which websites are different consumer segments likely to visit?’ Where are consumers more responsive to targeted ads (e.g. on a recipe site for a food product) and where are they best exposed to our brand (e.g. with a brand building message) without taking direct action? Key issues in building such a system included safeguarding data quality and integrity, establishing and enforcing proper use (respecting both consumer privacy and company ownership of datasets) and effectively communicating the benefits of the data to answer specific business questions. The end result was a searchable catalog, relating types of business questions to types of data, and suggesting standardized analytics with flexible recommendations.
What are your experiences with data science and market research? Do you agree with the approach outlined in this post? Let us know in the comments! And please join us for tonight’s presentation by Prof Woodrow Hartzog on ‘Privacy’s Blueprint: The Battle to Control the Design of New Technologies’, 6-7pm at Northeastern University, Hurtig 224.
Smarter Marketing gets Better Results 