Big Data?

July 24, 2013

 “Big data” seems to be the latest flavor of the month concept. We hear it used in context of the NSA, Obama Care, and cloud computing (which is not really done in the cloud, but in places like Utah and Norway).

The question is “does Big Data have anything to do with competitive intelligence?” Yes, but probably not in the way that you think. CI is certainly the beneficiary of the improving ability to collect and process more raw data. But, unless you are an arm of government, the data your firm usually processes is your own. That means, for most businesses it is inward, not outward looking.

CI is outward looking. That is not to say that the data that you need is not within your company – quite often it is. But is unlikely to be a part of its Big Data operations. Those typically deal with finance and production issues.

But Big Data has an impact on CI in indirect ways.

First, as companies can find out more and more about their internal activities at a greater level of detail, they have a tendency to expect that the competitive intelligence operations will be able to produce such detail. That is not likely since the number of people who have access to that detail at a competitor is limited, assuming the competitor has even bothered to generate that kind of data.

A second impact is that people increasingly believe that, somehow, CI can be conducted by computers, just by applying greater and greater computing power. Until computers are able to draw inferences from incomplete data, which they cannot do now, they will be of little impact to CI, other than to speed up and expand the amount of data that is collected. This is not always a good thing. More data means more time and effort spent processing it and a greater likelihood that the data contains within it increasing amounts of noise. That means time and effort wasted.

Competitive intelligence’s strength has always been in the analysis, not in the data collection. The world of Big Data does not provide CI with any help there; if anything, it hinders CI.

Thoughts on the SIR meeting

October 19, 2012

The recent annual meeting of the Society of Insurance Research (SIR) focused on “Big Data”.  It was interesting to see exactly what that meant to the attendees and to competitive intelligence.

Many attendees came with the belief that Big Data dealt only with issues like cloud computing.  But they left with a realization that they have to deal with issues including being flooded with more and more data, having  to deal with more and more unstructured data, and curiously, learning  to throw away data, early and often.

So what does this have to do with competitive intelligence?  Actually CI is involved in this in several ways.

First, Big Data means that more people in business, government, and the non-profit world are going to have to learn to deal with more kinds of data in more contexts.  That, in turn, means that they are going to have to learn at least a little bit about CI.  They need to understand enough about it to do a little of it, and to understand when they should get help, or to transfer an interesting bit of data to someone else when he/she can use it.

Second, it means that, for those doing CI, there are more people in more places within your enterprise that you can approach for least a little bit of help doing your work.  Now, if the push to get rid of “noise” in the data continues, you may be limited in time as to when people still have useful data, but it should change the way you look at developing and maintaining your own internal network.

Third, it means that people within your company will be gathering more different types of data from more different types of sources than in the past.  For those involved with CI, full-time or part-time, this means you have to rethink what kind of data you can possibly use, and how you can use it.  For example, one presentation dealt with the coming use of visual recognition technology coupled with real-time advertising feedback.  The example given was you watching the World Series, and following a home run on your television with your eyes.  Once it is clear that it is a home run and where it will land, the television feedback would insert a virtual advertising billboard behind that location. That means that as your eyes follow the home run downward, you would be reading the new advertisement. While the technology behind this and other scenes somewhat awesome, and necessarily antagonistic to privacy, the fact is that if it can gather one kind of data, from a CI perspective, it may be able to gather others.

So, while Big Data may initially seem to be a subject of interest only for those who toil in the vineyard of quantitative data, for those of us over on the qualitative side, it represents a new world.  Whether or not that is a brave new world is as yet unclear.