Mining for Insights: Lessons from Our Ancestors
By Sarah Tarraf, Director, Analytics, Gongos, Inc. & Susan Scarlet, Vice President, Strategic Branding, Gongos, Inc.
When it comes down to it, the centuries-old practice of mining can be best described as extracting raw material so that it can be transformed into something precious. It’s no surprise then that the metaphor of the Gold Rush has been applied time and time again to characterize the era of Big Data.
The voracious appetite for the “get” of data has all too often overshadowed its purposeful utilization—with a relentless focus on the acquisition, rather than the application. Rarely are there crude minerals pulled from the ground that don’t require distillation and refinement by scientists inside a laboratory. This is not just about panning for gold, folks.
Within the analytics space, there has been an underlying, ongoing assumption that data-driven insights will naturally jump out and show themselves. The critical process of deliberative exploration is often overlooked with large or ‘hard data’ brought on by the onslaught of big data. Whereas with small or ‘soft data’—the stuff of which primary research is made—there is a long-established practice of dedicating significant time and resources to such exploration. The well-trotted “path of discovery” cultivated by the market research discipline provides a blueprint worthy of attention by data science professionals.
So how do we borrow from critical and relevant market research standards to evolve to a more purposeful exploration and application of data in this so-called Gold Rush?
To illustrate this, we’ve examined the parallel and complementary paths of these two disciplines by looking at three critical stages of the insights process; and how each stands to better inform an evolving industry as it witnesses its very own type of convergence.
Data Collection
To remain competitive and relevant, more and more organizations are moving from mass marketing approaches to custom channels. Yet, to inform more personalized marketing messages, product refinement, and brand strategies; there is still a proclivity to cast a wide net on as much data as possible—both cross-channel and cross-platform. Moreover, there continues to be a striking disconnect between analytics departments that house the data, and insights teams charged with informing the lion’s share of business decisions.
This approach, however, runs counter to a primary research mindset, where the required data is openly articulated at the onset of the project. This is both the advantage and Achilles heel of primary research teams, yet for our example we will focus on the advantage. The benefit of using ‘hard’ data to inform decisions is that there is simply no shortage of it. However, with an undeliberative mindset you run the risk of draining time and resources when the data is not germane to the business issue at hand.
Insights leaders of the future will need to work hand-in-hand with their analytics counterparts to gain a deeper understanding of available and applicable data. This will enable more strategic and purposeful planning: in the short-term, using the data they have; and in the long-term, acquiring the data they need.
Data Exploration
Unlike traditional market research, Big Data exploration is often ‘black boxed’. All too often, this shuts key business users out of the process, thereby reducing strategic input, guidance, and gut checks along this critical path.
This is widely a technological roadblock. Users who lack the specialized statistical know-how to delve into prohibitively large and complex data files cannot participate in the exploration of mid-term findings.
Years of market research have cultivated steps and measures that provide business users with familiar and user-friendly mid-term results. In fact, peering into inbound, untranslated primary data is both possible and commonplace. This more systematic way of exploring the data, and sharing out the results, brings about the transparency necessary to include decision-makers in the process.
With hard data, this transparency is more difficult, but not impossible. Although the sheer amount of variables makes data tabulations overwhelmingly large, interactive simulators and dashboards enable business users to easily detect important and interesting relationships in the data. This allows them to explore and manipulate the data to further guide forthcoming analysis.
Data Application
Within market research analysis, both the data and the exploration include the input of stakeholders along the way. This leads to analysis that is purposeful, and ultimately both addresses and informs targeted business decisions. In the world of big data, the standards for a successful analysis can be somewhat different, often guided exclusively by the data analysts and scientists. Consequently, this yields analyses that may be highly predictive, but not prescriptive.
Data analysts are trained to assess statistical models based on set statistical criteria, whereas business users require models with variables within their span of control. As George E.P. Box suggested, “all models are wrong, but some are useful.” Often, diligent data analysts overlook this truism, while decision-makers keep it at the forefront of their minds.
Bringing stakeholders in at the onset and throughout the process has the added benefit of ensuring that model inputs are not only applicable, but actionable. These inputs should not only accurately predict the desired outcome, but also provide users with sufficient ‘levers’ to pull in order to achieve that outcome.
In closing, we must find ways to build “business user-friendly” laboratories that forge data analysts and decision-makers to work in tandem in the distillation and refinement of data. Under this scenario, “data miners” will move from copiously panning for gold, to gleaning priceless insights.
While big data analytics is the newest kid on the block – with significant investment dollars continuing to rush in – it would be best served to look to its “ancestors” in the consumer insights space to establish itself as a discipline that’s here to stay.
As published in Marketing Insights.