Asking the Right Questions

November 23, 2015

I just finished rereading a provocative book, Freakonomics[1]. When I finished, I realized it posited a situation similar to that raised by Superforecasting[2]. That is, to get an answer a problem, particularly a vexing one, you must ask the right question. And, as these books show, many times we do not do that.

What does that mean to those of us in CI? It means stepping back from “the problem” and clearing our head before we start our research and analysis.

Let me give an example from current headlines on terrorism. The US has bombing campaign underway against ISIS, and has been using killer drones to “decapitate” various terrorist groups.

Looking at them, it appears that they are the answers to the question “How can we destroy terrorist activity using aviation resources?” The answer is bombing and drones. Simple. But should the real questions be, instead,

Can we completely destroy terrorist activity using airborne resources? Have we or anyone else ever done that before?

Is decapitation (by any means) an effective way to destroy a terrorist group? When and where has that happened? Did it last?

What we have, it appears, is the generation of a military strategy based on Mark Twain’s observation: “To a man with a hammer, everything looks like a nail.” So, to a nation with the largest air force in the world and the greatest number of drones, eliminating terrorism looks like an air power problem.

This blinder effect is not limited to governments. It exists, to a great degree, in the business world as well. Take for example a question that might occur to you: How can my firm increase its market share in our largest product sector without reducing profit margins? Wrong question – you have already assumed away several significant questions:

Should my firm increase its market share? Is there a risk of increased anti-trust supervision, or of betting the company’s future by increasing its reliance on one product sector? How are those VHSs working out for you?

Can we actually hold profit margins at the current level? Are they now artificially high? Is this a market that is trending towards commoditization, so that profit margins will inevitably decline, making any quest for steady margins a fool’s errand?

The lesson for those of us in CI is clear: by carefully articulating and then methodically reviewing the questions we ask (or are asked by others), we can and must avoid (or at least diminish) the impact of our own and our firm’s built-in blinders in our research and analysis. Remember, by asking the wrong questions, you will never get the right answers.

[1] Steven D. Levitt and Stephen J. Dubner, Freakeconomics: A Rogue Economist Explores the Hidden Side of Everything, William Morrow, 2006 (Revised and expanded edition).

[2] Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction, Crown Publishing, New York, 2015, which I recently discussed at


November 19, 2015


There will be no blog this week in light of the tragedies in Paris. Pray for the victims and their families and that our leaders will do everything they can to protect all of us.

Can You Really Do Long-range Forecasting?

November 9, 2015

I have been increasingly critical of the validity of long-term (5-10 years in the future) strategic intelligence analysis and forecasts.[1] I just finished reading an important new book, Superforecasting: The Art and Science of Prediction[2], which makes me even more skeptical.

In that book, the authors review the results of multi-year, multi-participant studies, funded by the US intelligence community, aimed at determining what, if anything, makes any individual a “superforecaster”. The results are startling – in one way, and predictable – in another. I will deal with this in future book reviews.

But, with respect to long-term forecasting, the authors are devastatingly frank that this long-term forecasting is virtually impossible (my words). In their words,

“Taleb[3], Kahneman[4], and I agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious – ‘there will be conflicts’ – and the odd lucky hits that are inevitable whenever lots of forecasters make lots of forecasts….In my…research, the accuracy of expert predictions declined toward chance five years out. And yet, this sort of forecasting is common, even within institutions that should know better.”[5]

The book draws our attention to a classic real-world example of this – an internal US Department of Defense memo dealing with the then coming 2001 Quadrennial Defense Review, which looked at the strategic situation at the start of each decade between 1900 and 2000[6]. The authors dryly note that “in every case, the reality was a stunning change from ten years earlier.”[7] Oh, this memo was written exactly 5 months before 9/11.

To be fair, before accepting my view, you should consider whether or not my selection of this analysis is just a case of confirmation bias.[8] (VBG)

[1] See, for example,,, and

[2] Philip E. Tetlock and Dan Gardner, Crown Publishing, New York, 2015.

[3] Nassim Talib, former Wall Street trader and creator of the “Black Swan” concept.

[4] Nobel Prize winner (economics) Daniel Kahneman.

[5] Op. cit. pp. 244-45, emphasis added.


[7] Op. cit. 242.

[8] Op. cit. 38-39.

Securing Off-site Meetings

November 4, 2015

The popularity of holding off-site meetings comes and goes. In some cases, their use is designed to bring together people from offices or locations that don’t normally have physical interaction. In other cases, they serve to enhance team-building. In yet others, it is to provide a measure of security not available at a company’s regular offices for matters of some sensitivity.

There are a number of simple steps that should be taken at off-site meetings to prevent the accidental release or purposeful capture of sensitive or confidential information, whether to competitors, the media, or the public:

  1. Find out who else has meetings at the site you are considering using. While you may secure your site, when your people are taking a coffee break they start talking in the hall and others may overhear their conversations. You should do this at two points: first when you’re considering retaining these site and second, just before you go there to see if things have changed.
  2. Make sure you check everyone that comes into the room. Outsiders can wander in “accidently”.
  3. If you are going to have a registration table and/or display table, consider putting it inside the room or rooms you’ll be using rather than the hall. If you place it in the hall, then you need to have somebody of the table at all times to keep the materials secured as well as to keep from prying eyes things like attendance lists, notations of incoming calls, etc.
  4. If you are distributing materials at the meeting, distribute them that the meeting, not before. In fact, distribute them in the conference room, in public halls the hotel or convention center.
  5. Clearly mark all materials as company confidential, proprietary etc. This will not stop some people from taking these materials, but will discourage those who operate on an ethical basis. It also should alert your attendees to be careful with them.
  6. Remind the people there that what you are doing is confidential, and is not to be discussed outside of the meeting rooms, including in the halls, at the bar, the pool, on the golf course etc. No discussion outside of the room means no In addition, remind them that any materials you hand out are to be handled with care. If it is a very sensitive matter you may consider having people leave materials in the room and locking it at the end of each day.
  7. Keep communications in the room secure. Have all attendees turn off all smart phones and tablets. That is aimed at keeping attendees from recording the proceedings or taking pictures, as well as communicating with outsiders. If that is not possible, ask that these instruments be put in airplane mode, so that no incoming or outgoing calls can be made. This also cuts down on distractions.
  8. When you are done with the meeting, sweep the room – yourself. Do not rely on the hotel staff for this. That means collecting all materials and notes left behind, wiping all whiteboards completely, and removing all trash from trash cans that have been the depositories for conference materials. Securing a room during the meeting and then leaving copies of the agenda with a whiteboard showing conclusions reached on a new marketing campaign is not security – it is folly.

By the way, if you’re holding the meeting on-site and it is a sensitive matter, the same cautions apply.