Current and Future Problems for CI

September 19, 2018

Over ten years ago, I spoke to a SCIP chapter in Atlanta about some of the major problems I saw in CI, now and in the future

Here are the notes for that discussion.


Current Major Flaws in CI


Most CI staff are forced operate by looking into a rearview mirror. That means that management rarely gets a view of where they are going, but rather of where they and their competitors have been. For example, virtually all of CI’s current processes, including in particular the use of KITs and KIQs, are backwards looking and reactive. Ultimately, they lead to the production of CI with less and less value. And that makes those providing the CI of less and less value.

  • Solution: Phase out KITs & KIQs. Let CI staff be freer to anticipate needs, to work directly with key end-users.

Many CI collectors and analysts are being forced to use less and less primary sources by a variety of technological and legal restraints. But, this increased reliance on secondary sources actually tends to support a growing image of CI as akin to library work. And that image may be right!

  • Solution: Get clear guidelines/ethical statements affirmatively allowing for primary research; point out where lack of primary research, interviews, attending trade shows, etc. is constraining CI. But don’t whine!

Too many efforts to integrate CI with market research fail. In fact, these efforts are almost always doomed to fail. Why? Because CI is qualitative and MR quantitative. Most MR managers cannot deal with what they see as a lack of precision in CI. And most CI analysts cannot understand the obsession with “numbers”.

  • Solution: The very few successes have occurred when the head of the combined unit either (a) comes up through CI or (b) has been a “customer” of CI in the past.

Your client knows more each day, or at least thinks he does. But what he knows might not be right.

  • Solution: Read what your client reads and get critical CI to her before she asks for it. If trade sources are wrong, make sure to point that out!

CI as we know it today totally fails to provide any meaningful tools for the next generation of managers who expect, or will be expected to, do their own CI.

  • Solution: Wait a few minutes.

Fatal flaw – not all intelligence has to be actionable. Generating real understanding in a clear context can be critical.

  • Solution: Increased education of end-users and decreased use of KITs, etc.

Some Failures That Will Emerge Due to Reliance on the “Government Model”

First, a question: According to some, intelligence analysis in government is still not a true profession. Why should we assume it is in the private sector? Just a thought?

The Government Model – the black box after 5 decades. Based on looking at a few targets, most of which generated external “objective” data, coupled with access to secrets, often internal deliberations.

Now being challenged for several reasons:

  • Too many end users
  • Too rigid in needs determination – Sound familiar?
  • Too driven by the reports to be given instead of the intelligence on questions yet unasked out there
  • Still believing that secret data is more useful than data in the public domain – essentially dissing analysts!
  • Focus on the short term, not the long term
  • Focus on what the other side does, not what they think or what they feel
  • Too much demand for certainty; drives out dissent in name of unanimity
  • Designed to focus on one or a few targets; not a functional model in an era of multiple, changing targets.
  • End users also get data, good or bad, from other sources. Intelligence no longer the sole source anymore. Again, sound familiar?
  • Buying into the end users definition of needs begins a buy-in process of their biases, their politics, world view, etc.
  • Analysts are too far removed from end users of intelligence, with exception of military/tactical, that is battlefield intelligence.
  • Lack of experienced analysts to train the next generation of analysts. Current training lacks hands-on “interning” type environment that is most helpful. Also lacks the ability to test analysis of the analysts.
  • Inability to deal with “too much” data. Just what do they do with all of that take from the NSA? Welcome to the Internet Age!

These same failings are occurring or will occur in almost every case for the private sector. Why? We have adopted, or at best adapted, a flawed model.

Biggest Future Unmet Needs

Above all of this is a problem not yet recognized by CI. That is the problem of success.

In the past 25 years, the most common model was a formal (or informal) CI unit, charged with collecting data, generating analysis, and communicating the finished intelligence to an end-user. In some case, the unit was one person, serving as both collector and analyst; in others, it was a dozen or more people, dividing among themselves collection and analytical functions.

CI is finally achieving its goals: incorporated in graduate schools, understood by other disciplines, talked about at the AMA [marketing], MRA [market research], SLA [libraries], AIIP [information brokers], LMA [law office managers], PDA [product development], ASIS [security].

In 1986, Carolyn Vella warned that CI could go the way of strategic planning. She meant the over promising, over bureaucracy, etc. that marked SP at that time. At it turned out, strategic planning’s association imploded as planners were replaced by executives telling manager to do their own planning. CI may be close to that position.

Now, as CI becomes integrated into other business processes, a new model is emerging – not easily. That is one where the collector, the analyst, and the end-user are all the same person. CI’s present models and processes do not fit that new archetype, whether it is in the areas of ethical conduct, needs determination, communication, data collection, or utilization.

The CI cycle and all of the literature surrounding it no longer applies – Needs, Collection, Analysis, Dissemination, Utilization.

What about ethics – where is the check on unethical means of collection?

  • Revise ethical policies to acknowledge the incorporation of CI into general management. And really train everyone on them. Yearly, if needed.

Where is the pushback in defining needs more sharply when you do it yourself?

  • Stop before you start collecting, even though you have already been doing it. Recall your time as a student. Realize that collection and analysis are now not merely linked – they are merged!

Where is the need to write up a separate analysis when you just synthesize it into what you are doing?

  • It is good practice to separate what you know, suspect, and yes, guess, in a report or presentation. You are not God. Don’t deliver a message on stone tablets!

Where is the review of what you did before you present it to be used?

  • Get someone else to at least read it – critically, really critically. If you cannot deal with that thought, your work product probably could not stand up to it anyway.

Where is the completion of analysis when the data is just absorbed as it comes in?

  • It is not a bad thing that the analysis is continuing – changes in the competitive environment don’t stop for your Monday briefing, do they?

Where is the feedback in terms of success of the intelligence work?

  • You had better learn to evaluate what you do poorly – you know what you do well.

Where is the ability (or is there a need) to justify the costs of getting the intelligence?

  • If you cannot use what you collect, why are you bothering? Develop new targets, new data sources, new tools!

Where is the application of a variety of analytical tools? To a man with a hammer, everything looks like a nail!

  • This is a problem now for analysts. In most cases, you only have to decide if you are assembling a puzzle or proving/disproving a working hypothesis. Remember, if you think it is only working with a puzzle, how do you know the pieces you have are even from the same puzzle!

What is the relationship with the library/information center process?

  • They can and should become your first stop in collecting data. Even if they do not have the data you need, they can help identify where/from whom you can get it. They should become more valuable, not less.

What kinds of skills are needed? The current CI research indicates that there are a variety of skills needed to be a CI analyst. The odds of having them in one person are nil. In fact, a parsing of these skills divides them into the Nero Wolfe-Archie Goodwin divide. If they are hard to get in one person now, how do you add them to a batch of OTHER skills/training a manager must have?

  • Here, you will have to just make do. No one has all of the skills that they need. Just work at acquiring new ones and polishing old ones.

What this all means is that without a new CI process aimed at the end user as analyst and/or collector, they will misuse CI, produce poor CI, or none at all. At best they will be subject to what one critic of US intelligence has recently called, and I paraphrase, the spectacle of the collection of factoids driving out real thinking.

Note: Oddly enough, this disconnect does not apply to the outside source of CI. There, the current CI model would seem to work perfectly well there, with the exceptions noted earlier. And, based on our experience, as well as the facilitated sessions at SCIP06, company restrictions, such as above, may actually drive work towards outside consultants.

What are the overall solutions for CI to avoid imploding as it achieves success?

In no particular order:

  • Admit that CI is not a profession. Lawyers still refer to their practice, a “guild” term.
  • Redirect training at SCIP and by private consultants to training non-CI professionals in CI. Already those in security are looking at CI (defensive) as a new tool. Others will too, and soon.
  • Train all managers who might conduct or use CI on what CI really is. Stress the concept not the process. CI is a tool for sales, marketing, crises management, and strategy. It is also a tool for human resources and many other functions. Recognize that and deal with it. The Baldrige Awards already have – for years!


Looking back, I do not see much progress on any of these, do you?

Emerging Problems for Elicitation Interviewers

May 30, 2018

A recent newspaper article[1] discussed the ongoing “death of voicemail”, particularly in the case of millennials. Now, I have previously pontificated on some the difficulties of communicating with millennials, Millennials and Competitive Intelligence (2 parts)[2], but this takes it to a new level.

If the trend this fascinating article describes holds, the death of voice mail will have a significant impact on competitive intelligence research’s elicitation interviews. One key element of that is that you, the researcher, are able to get access to someone you have never met, whose business email and/or personal email accounts you do not know, to talk with them.

Consider these quotes from this interesting article:

“When people leave me voice messages, I just delete them without even checking. If they want to get hold of me, they can text me.”

“This is a large generalization, but they [millennials] don’t feel that comfortable in face-to-face spoken interaction or its derivative over the phone.”

“In the last three to five years the majority of phone calls in my world are booked ahead of time, just like a meeting.”

“Fewer and fewer people are going to have that skill [talking on the phone].”

As an aside, the people and patterns described in this article do not bode well for those very millennials described therein. Why? Ok, how do you get to know new people in your own business or neighborhood or graduating class when you only respond, by email or text, to a message on your voice mail (but never listening to the voice mail), only so long as you already know that person’s phone numbers and email addresses. A very static circle, isn’t it?

I wonder what would happen if the EVP called one of these people to ask questions about a report he/she did, and never received a reply because the employee did not recognize the EVP’s cell number, so he/she just deleted the message without listening to it. Think about it.

[1] ETHAN BARON, “How the death of voicemail is changing the way we connect”, The [San Jose] Mercury News, May 13, 2018.

[2] and

Long Range Thinking

April 25, 2018

When researching long-range trends, or more accurately spotting long range trends, you should seek out sources that (a) are thinking about the future and (b) have had some reasonable success in identifying such trends.

I recently came across one. Published in 1997, it correctly predicted a surprising number of developments. Here is just a partial list:

  • Development of smart clothes and spam filters.
  • Stagnation in middle income salaries due to the Internet producing more candidates competing for jobs.
  • The use of complex comparisons in advertising to prevent, or at least slow, price competition among providers of virtually identical products\services, such as phone, financial, and insurance service
  • Marketers use of the Internet and your computer to track your interests and activities so that their marketing can be ultra-focused.
  • People giving up privacy for more security to the point that there will be cameras virtually everywhere.
  • The corollary that “everyone will be a news reporter”.

Of course, this sage was also wrong, for example, in predicting that “airline travel will be just as uncomfortable as it is today [1997].” As we all know, it is WAY MORE uncomfortable.

What is the source of such profound insight? Was it a futurist, global consulting group, a recently declassified CIA study, or 1997 translation of Nostradamus? No.

Ready? It was Scott Adams, best-selling cartoonist[1].

You should not be surprised as humorists as rule are a very savvy, observant group. It was said of long-time Tonight Show host Johnny Carson that you knew which politicians were in trouble just by listening to his opening monologue.

So, when you are seeking help in figuring out long term trends, never prematurely narrow your search – err on the side of broadening it.

[1] The Dilbert Future – Thriving on Stupidity in the 21st Century, HarperCollins, New York, 1997, pp. 31, 48, 132, 160, 171, 198, 202, and 221.

Generalist versus Specialist

April 3, 3018

What headlines in the health insurance industry! It is an industry which has traditionally looked at itself as relatively protected from outsiders. (Remember the concept of “barriers to entry”?) First, Obama Care turned the individual market upside down and may have threatened its very existence. Then, there is massive change is coming or pending due non-insurance firms including CVS, Walmart, JPMorgan Chase, Amazon, Berkshire Hathaway.

Question – did the competitive intelligence teams at the major health insurance companies foresee this sea change and warn their management? I do not know for certain, but having spent time there, I am guessing not. Why?

I think that the health insurance industry, like too many others, erroneously favors experience in the industry over CI experience/training in its CI providers, both inside and out. And that preference for industry specialist over generalist is widespread.

Let me give an example. Some time ago, a head hunter contacted me looking for a candidate to fill a slot in another “health” industry. The client’s detailed specifications required x years of direct CI experience in that industry – only. The client was willing to drop back on time in CI, but not in time in the industry. It was non-negotiable. I told the recruiter that there were not just very few people that met that standard, but in fact there was only one. And that person I knew was soon retiring. I pressed, and soon learned that the headhunter’s client was the very firm where that person worked.

I told the recruiter that I knew of many excellent candidates with extensive CI experience, but their industry experience was in related industries. The recruiter replied that the client was adamant. So, the client ultimately found no one, by ignoring more general experience in favor of specialized industry experience. Over time, the CI unit basically dissolved.

Sometimes generalists have it over specialists. Listen to the late Joseph Campbell, world-renowned expert on myths (and regarded by some as inspiring the Star Wars sagas):

“Specialization tends to limit the field of problems that the specialist is concerned with. Now, the person who isn’t a specialist…sees something over here that he has learned from one specialist, something over there that he has learned from another specialist – and neither of them has considered the problem of what this occurs here and also there. So the generalist…gets into a range of other problems….” [1]

[1] Joseph Campbell with Bill Moyers, The Power of Myth, Doubleday, NY, 1988, p. 9

Got Style?

March 23, 2018

Writing style is much overlooked (note the unfortunate, but common, use of the passive voice here) when we talk about business communication. However, what we write survives longer than our oral presentations do. A written document can always be referred to after the fact, while a presentation, unless recorded, relies on (imperfect) memory. So, your style of writing is important.

And, trust me, there are different styles of business writing. Here are a couple:

  • The entertaining – conversational, relaxed, and short, but tends to be lighter on content. It is an opening shot for discussion.
  • The direct – short sentences, clear language with no passive statements. It offers an issue and a conclusion. It reads like people (should) sound.
  • The professional – longer sentences, with more technical terms, presumably included for precision. No one, or almost no one, speaks like this – unless of course they are just reading it aloud, which is whole different issue. Then it becomes merely boring and unintelligible.
  • The overbearing – involves complex sentences, heavy use of acronyms. It is aimed at convincing the audience that the author is a real (and perhaps the only) expert on the topic. It is designed to persuade by being overwhelming, including the excessive use of footnotes and/or quotations.
  • The political/bureaucratic – filled with refences to rules, regulations, “context”, and past actions/decisions. Operates to conceal and deflect, by using the passive voice, rhetorical questions, and deep dives into often irrelevant sidebars. Rarely includes any acceptance of the possibility of (a) personal error, (b) institutional failure, or (c) cogent opposition to its conclusions.

Which are you (and which is this)?

Welcome to the [R]evolution

March 5, 2018

“Got a revolution, got to revolution.” Jefferson Airplane, Revolution (1969)

in our new book, Competitive Intelligence Rescue: Getting It Right (Praeger 2017), Carolyn Vella and I relate a case dealing with DIY CI (chapter 8). . Let me give you a couple of my thoughts on DIY CI.

Remember that the CI universe today has three basic research and analysis epicenters:

  1. CI professionals within an enterprise (including adjuncts such as researchers sited in libraries/information centers)
  2. Independent CI professionals who consult for/research for that and other enterprises
  3. Internal DIYers.

My own perception is that the first group is static or growing slowly, the second is stable or slightly declining, and that the third is growing steadily. Compared with 10 or 20 years ago, the existence of DIY CI marks an important evolution, if not revolution, in CI. Those growth trends, if they continue, may fundamentally change the CI “business”.

One plus from this is that it shows an increasing use of CI in enterprises, coupled with better access to end-users, particularly since the end-user in DIY CI is the person who generates the CI. It should also mean that the time between a perceiving a need for CI and its creation could fall.

However, there are also some minuses:

  • Those producing the CI will necessarily have narrower experiences in producing it, since they deal only with one client. That could result in a loss of professional perspective or even the failure to develop it.
  • The use of elicitation interviews will necessarily fall, thus diminishing use of a proven, valuable primary research resource.

What does this mean? One consequence could be that CI degenerate into several subspecialties where experience and developments are not easily transferable, such as IT CI, pharma CI, B2C CI, etc. Another consequence could be that CI could morph into a discipline that will not be able to look forward as easily as is it can look back and look at the present. Why? Because data on future actions and intentions lies with people to a significantly greater degree than in published sources. A third could be the separation of early warning processes from everyday CI, in part due to the lack of necessary broad perspectives among internal personnel.

What to do to keep these trends from “damaging” CI? (Sorry, I know that is a loaded question, but that is how I see it):

  • Institute regular awareness sessions and focused training both on producing CI and on using it. To avoid inbreeding, vary the sources for that training. That is use insiders, then external resources, and vary the outside providers over time.

Establish a stable of outside CI professionals pre-approved for future assignments. Rotation among them avoids having them buying into your firm’s blinders. Also, use one or more of them to regularly review your CI processes and work products to enrich your program with their broader perspectives. Interestingly, this is a flip on the CI audit that was used in the early days of CI before initiating a new CI program. Now the audit would be of the system as it operates and not of the potential need for CI and existing internal resources

Sage Advice – Strategic Planning and CI

January 17, 2017

How do these observations apply to competitive intelligence?

From General (and former President) Dwight D. Eisenhower: “Plans are worthless, but planning is everything.”[1]

From legendary science fiction author Isaac Asimov: “To succeed, planning alone is insufficient. One must improvise as well.”[2]

They mean that planning of any sort must not end with the plan. Being a part of the process means learning to work with all others similarly situate, even after the plan is done, to take necessary action based on accurate intelligence. You and they must be constantly alert to all of the changes that will impact the plan, or even render it useless.

You may be asked to work with strategic planners and provide CI, whatever it is called. Keep in mind that the need for such intelligence never ends at the “completion” of the planning cycle. Whether or not recognized by the planners themselves, CI is even more vital to the entire planning and execution process once the plan is done.

Why? Your competitors and the competitive environment will not give you the luxury of staying static or doing what you expected them to do in response to your firm’s actions, just to accommodate your planned efforts. Stay linked to those throughout the process and focus on keeping them updated rather than waiting for the next cycle or even the next updating meeting.

[1] Quoted by Dean James Stavridis, Fletcher School of Law and Diplomacy, in “Trump’s national security strategy leaves too much unsaid”, Time, January 22, 2018, p. 45.

[2] Isaac Asimov, The Foundation Trilogy, Ballantine Books, 1983, p. 126.