Why Data & Diamonds Are An Analyst’s Best Friends: blog + video
by Nick Wright, Data Analyst, Gongos, Inc. & Sarah Tarraf, Data Analyst, Gongos, Inc.
It was Aristotle who said: “A friend to all is a friend to none.” If that is the case, then Big Data has few friends, if any, with its attempt to befriend us all.
Big Data is the Holy Grail of analysis to statisticians; it is the promise of smarter, faster decisions to the business executive; and it is a good source of job security with the seemingly never ending need for extracting, transforming, and loading more data to the IT professional,.
To clarify what Big Data means to each of these players, we see other terms popping up—some trademarked, some not—in an attempt to cut through the noise. Today’s jargon of choice might be: Hadoop Infrastructure, MapReduce, Cloud Computing, Business Intelligence, or ‘The Internet of Things’. Unfortunately to the analyst in an organization tasked with doing Big Data analytics, these add little to no clarity as to where the focus should be placed. Where are the real benefits of Big Data found inside an organization—and as such, where should a diligent analyst focus his or her energy?
Within Gongos, Inc.’s analytics teams, this is a question we continue to wrestle with, i.e. should we focus our energies on IT functionalities that allow us to harness, store, and manipulate data? Or should we concentrate on the advanced mathematics used to extract meaning from data? And if so, how does this differ from traditional analytics – “Small Data”, if you will.
Time and time again, throughout our discussions, a common metaphor has given rise to great clarity among our team. As metaphors go, let’s consider the data itself as a raw material.
We’re currently in the midst of a “Data Rush”—corporations are flocking to collect as much data as they possibly can, enabled by previously unknown IT functionalities that allow Big Data entrepreneurs to gather and house unprecedented amounts of information.
In the early stages of this revolutionary rush, data for data’s sake actually was very valuable. However, we find ourselves currently in the midst of an inflection point. The ‘gold flakes’ of data collected for data collection’s sake are now table stakes. It is no longer a question of how we will house and transform the vast and often messy amount of clickstream, social media, and consumer purchase data—but what higher order decisions can this data inform? The real questions are now…What insights is this data generating? What business decisions are they guiding? In other words—corporations now no longer just need flakes of gold—they are seeking the fully formed 24-carat wedding band.
Enter the decision scientist.
In our metaphor, the jeweler is the Decision Scientist—the individual who transforms the raw material that is data into priceless insights. While we’re bombarded with blog posts, articles and sales pitches directed at the ‘data miner’ and his or her respective tools (the Hadoops and NoSQL Databases of the world), we hear little about how to move past these so-called tools and extract higher order value from the data.
It is becoming clear that while we have an inundation of information and resources in the ‘data mining’ space, there is substantially less focus in the ‘insights generation’ space. That is, we need fewer miners and more jewelers.
Thinking in this way, we realize as analysts it is important to have knowledge of the data extraction space – the table stakes must be met. The truly revolutionary wins driving this inflection point, however, are leveraging the data at the next level (think NetFlix’s Algorithms, Amazon’s Recommended Items Engine). They have moved past delivering a bag of gold flakes to crafting a set of beautiful gold wedding bands. The new decision science-driven analyst must learn to focus more energy on the step that comes after the extraction of the data—finding the actionable insights that lie within that data.
Next up, the analyst must leverage techniques in many of the same ways as traditional or ‘small’ data. However, the payoff promises to be much greater. It may take pounds and pounds of gold flake to develop a rare and valued piece of jewelry – and the new “Big Data” age allows for more raw material than ever.
As today’s and tomorrow’s analysts, we’re lucky to function in an environment with a deluge of data. However, as Decision Scientists, we must learn to put that material to its highest order use —fusing mathematics, statistics, and business knowledge to distill a vast amount of unintelligible data into a distinct set of actionable insights with implications.
Within this new, data-rich world, we as Decision Scientists, must place our priority on the identification of insights—paying attention to the much-hyped technological aspect of Big Data, but not overlooking the proficiency and dexterity that ultimately lead to better decision making.
We must strive to be the expert jewelers in this new hyper-data system—leveraging new technologies to help our clients get a real seat at the table. In doing so, we cannot let go of the creativity, the challenge we love, and the craftsmanship required to deliver (and deliver well) in this new era of data driven insights.
[To receive a quarterly email of our latest thinking, subscribe below.]SUBSCRIBE