10 years ago
Leveraging Dark Data: Q&A with Melissa McCormack

Melissa McCormack,Research Manager at predictive analytics research firm Software Advice, was recently interviewed about dark data by our Event Marketing Manager, Crystal Prag. Learn about businesses that can benefit from dark data extraction and processing, and also discover how businesses can continually merge dark data and traditional sources of light data.

What ways can businesses begin to merge dark data and traditional sources of light data on an ongoing basis?
Evaluate the data you currently collect and pinpoint data sources that may correlate and/or add value to those data. It’s important to be aware of what light data you already have, and how it can answer business questions, before probing into the dark data.

Pair relevant groups of data with each other, keeping in mind variables that are typically correlated or that you have observed are related after analyzing the data you have collected.

Eliminate irrelevant data sources from your analysis (though you should still collect such data in case it may come in handy down the line). In essence, to streamline the analysis process, only use sources that are relevant to issues at hand.

What are some other major areas in which dark data is being underutilized besides underutilized customer information?

Education and Healthcare come to mind. For-profit businesses tend to see the value of dark data right away, whereas it’s less obvious or immediate for the nonprofit sector. But imagine if these two sectors were mirrored after the customer service or financial service industries in regards to the way they use data to make businesses decisions. The potential to service students and patients in the manner in which the consumer and financial services pursue their target population is huge. So much paperwork is involved in both education and academics, so the data is there—and in the age of electronic health records government incentives, much of it in the healthcare space is now digital. However, it needs to be mined and analyzed in order to lead to opportunities that effect the change which usually results from the strategic use of personal and behavioral data.

What kind of businesses can really benefit from dark data extraction and processing?

Really, any business that sells a product, service or idea—anyone who has customers—can benefit. The goal is to find explanations for, and even predict, shifts in your target demographics’ needs and preferences. Correlating traditionally dark data sources with light data sources is a valuable tool and is worth allocating the necessary resources toward.

How old is too old when it comes to dark data?

There’s no such thing. As long as you have the space and capacity to store data, you should, regardless of age. You never know when those unstructured call center notes from 1993 will come in handy. Creating a data library is invaluable, and it will only grow as your business grows. Having historical data will allow you to analyze relevant shifts and trends in demand, as well as predict and plan for the future based on what you’ve observed in the past. That said, if you’re analyzing, say, customer sentiment in social media, you simply won’t have relevant data that predates the advent of social channels. So in that case, dark data from before those channels existed could be considered “too old.”