Asking the Right Questions

November 23, 2015

I just finished rereading a provocative book, Freakonomics[1]. When I finished, I realized it posited a situation similar to that raised by Superforecasting[2]. That is, to get an answer a problem, particularly a vexing one, you must ask the right question. And, as these books show, many times we do not do that.

What does that mean to those of us in CI? It means stepping back from “the problem” and clearing our head before we start our research and analysis.

Let me give an example from current headlines on terrorism. The US has bombing campaign underway against ISIS, and has been using killer drones to “decapitate” various terrorist groups.

Looking at them, it appears that they are the answers to the question “How can we destroy terrorist activity using aviation resources?” The answer is bombing and drones. Simple. But should the real questions be, instead,

Can we completely destroy terrorist activity using airborne resources? Have we or anyone else ever done that before?

Is decapitation (by any means) an effective way to destroy a terrorist group? When and where has that happened? Did it last?

What we have, it appears, is the generation of a military strategy based on Mark Twain’s observation: “To a man with a hammer, everything looks like a nail.” So, to a nation with the largest air force in the world and the greatest number of drones, eliminating terrorism looks like an air power problem.

This blinder effect is not limited to governments. It exists, to a great degree, in the business world as well. Take for example a question that might occur to you: How can my firm increase its market share in our largest product sector without reducing profit margins? Wrong question – you have already assumed away several significant questions:

Should my firm increase its market share? Is there a risk of increased anti-trust supervision, or of betting the company’s future by increasing its reliance on one product sector? How are those VHSs working out for you?

Can we actually hold profit margins at the current level? Are they now artificially high? Is this a market that is trending towards commoditization, so that profit margins will inevitably decline, making any quest for steady margins a fool’s errand?

The lesson for those of us in CI is clear: by carefully articulating and then methodically reviewing the questions we ask (or are asked by others), we can and must avoid (or at least diminish) the impact of our own and our firm’s built-in blinders in our research and analysis. Remember, by asking the wrong questions, you will never get the right answers.

[1] Steven D. Levitt and Stephen J. Dubner, Freakeconomics: A Rogue Economist Explores the Hidden Side of Everything, William Morrow, 2006 (Revised and expanded edition).

[2] Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction, Crown Publishing, New York, 2015, which I recently discussed at https://diy-ci.com/2015/11/09/can-you-really-do-long-range-forecasting/.


2 Comments on “Asking the Right Questions”

  1. […] just finished rereading a provocative book, Freakonomics. When I finished, I realized it posited a situation similar to that raised by Superforecasting. That is, to get an answer a problem, particularly a vexing one, you must ask the right question. And, as these books show, many times we do not do that. What does that mean to those of us in CI? It means stepping back from “the problem” and clearing our head before we start our research and analysis.  […]

  2. tripkrant says:

    Hi John,

    You cannot get the question(s) right, without getting the problem right before hand. As the saying goes: “a problem well stated is a problem half solved.” Questions should be developed from target-network and problem definition models. This is of course is Robert Clark’s “target-centric approach” which is Intel Analysis 101 reading in most places, but apparently is too complicated for most CI folk.

    Clark’s approach is relevant because modern intelligence problems (such as terrorism) are , to use Gregory Treverton’s famous dichotomy, mysteries rather than puzzles. (See: “Risks and Riddles”: http://www.smithsonianmag.com/people-places/risks-and-riddles-154744750/?all ) That is, it poses a question that has no definitive answer, because it is contingent upon too many interacting variables (complexity) that are both known and unknown.

    Al-Qaeda is like slime-mold, you can’t really kill it completely – its just an idea – so you have to preempt it, fracture and subvert the network. We have killed dozens of AQ “number 3 guys”, and it didn’t really do anything because in a network EVERYBODY is a “number 3”. ISIS is a proto-state and has physical infrastructure that can be destroyed to degrade its capabilities – i.e. there are answers to some of its problems.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s