Security analyst Bruce Schneier’s post on the apparent failure of the U.S. government to stop Syria’s alleged use of chemical weapons holds lessons for Big Data and CIOs.
Security analyst Bruce Schneier’s post on the apparent failure of the U.S. government to stop Syria’s alleged use of chemical weapons holds lessons for CIOs about the value of information—more isn’t necessarily better, or useful.
That lesson is particularly relevant now, as CIOs are assessing the investment case for a range of new tools in Big Data and predictive analytics. Companies from across the economic spectrum are testing, and in many cases actually using predictive analytics, in the hopes they can boost their fortunes by being able to predict the future.
For example, a recent study from New Vantage Partners shows that more than two-thirds of financial services companies have at least one Big Data initiative under way, and that 33% have at least one Big Data project in actual production. Furthermore, 75% of these are spending more than $1 million this year on Big Data, and 94% plan to be spending at least that much by 2016.
But if U.S. intelligence services are any example, there may be limits to what companies can accomplish using these new – and in many cases, expensive – technologies. CIOs need to carefully consider what they’re promising when they argue for this type of investment.
Take what we now know about what we knew – or thought we knew – about Syria’s use of chemical weapons. As Mr. Schneier notes on his blog, U.S. intelligence agencies had “at least three days warning” that the Syrian regime was preparing to launch a chemical attack, but chose not to act on that knowledge. As Mr. Schneier also notes, there are several possible explanations for why the U.S. didn’t act on this knowledge. He says these reflect a “ fundamental problem with intelligence information and our national intelligence apparatuses.” But he may as well have said it reflects a fundamental problem with intelligence gathering and anyone’s ability to act preemptively.
CIOs are being sold on the idea that they can gather information in “real time” which can then be used by other systems to lure shoppers to spend more, or drivers to use more efficient routes. Predictive software, some vendors claim, can help retailers know that shoppers will buy shovels one day before a snowstorm is predicted to occur, rather than three days before or the day of the storm itself. But the question is whether they really needed software to predict that. And how much more accurate is the software than a seasoned veteran’s estimate?
Predictive analytics that measure machine behavior, rather than people’s behavior, might be more successful. Sensors can first predict when a ball bearing will wear down and then send a signal to another machine kicking off the replacement of the soon-to-be-broken bearing. Software can detect when another application is behaving in a way that indicates it will try to surreptitiously transfer data to a server outside the corporate firewall.
In the case of the NSA, Mr. Schneier says there are a number of explanations for why nothing was done about Syria’s use of chemical weapons: analysts may not have understood the data they were getting; they didn’t have enough information to be sure; they were sure but didn’t want to reveal their “sources and methods;” or that “there was nothing useful we could have done.” Leaving aside the politics, those possibilities should send shivers down the spine of CIOs heavily invested in so-called Big Data systems.
“More data does not necessarily mean better information. It’s much easier to look backward than to predict,” Mr. Schneier writes. He concludes:
“We’ve just learned from the recently leaked ‘black budget’ that we’re spending $52 billion annually on national intelligence. We need to take a serious look at what kind of value we’re getting for our money, and whether it’s worth it.”
CIOs should be asking the same questions of themselves before their CFOs or CEOs ask it for them.
By: Michael Hickins, Editor, The Wall Street Journal and CIO Journal
Originally published at The Wall Street Journal
 |
 |
 |
