Intelligence Analysis: Art or Science?


Check out this paper titled “Intelligence Analysis in Theater Joint Intelligence Centers: An Experiment in Applying Structured Methods” (I know, sounds like a real page turner, but keep reading, it’s worth it).

There’s a debate within the intelligence community (well, ok, debate is probably a strong word – discussion?  lone ramblings of a deranged mind?) about whether analysis is more like an art or a science. Depending on where you fall within that debate determines quite a bit about how you go about conducting analysis and how you view the ability of others to become analysts.

Of those who believe it’s an art (and I was firmly in this camp for a number of years), intelligence analysis is seen as the result, primarily, of imagination and inspiration. While subject matter expertise is important, good analysts were seen to possess some, intangible ‘X-factor’ which allowed them to look at data in ways mere mortals weren’t able to and ‘voila’ out would pop some hidden wisdom. I think there’s a number of reasons why that view has a lot of appeal for analysts and non-analysts alike. For analysts, it feeds their egos to be in a select group that can peer behind the curtain and see things that most cannot. Besides, books, movies and TV have, for decades if not centuries (think Sherlock Holmes) have fed into this view.

While that paradigm may have worked for quite a long time a host of factors over the past twenty years (the internet, the rise of the 24 hour news cycle, the rise of the ‘punditocracy‘ and, of course, my old nemesis Tom Clancy) have allowed virtually everyone to have access to the same information and led people to believe that with access to information, anyone can conduct analysis off the cuff.

This has proven a serious obstacle in both law enforcement and military circles. As Folker writes (and no, it’s Folker, not Focker)

Decision-makers typically form conclusions based on intuition developed through years of experience. Therefore, the intelligence analyst who provides an assessment based on inexperience and his own (lesser developed) intuition should not expect to be taken seriously.

In my own experience in Afghanistan, I found that decision-makers didn’t even need years of experience to form conclusions. In many cases an affinity for the TV show ‘24‘ and too much time watching Bill O’Reilly was sufficient to come up with analysis that would trump (at least in the mind of the decision-maker) anything an analyst could produce.

So, this problem of perception (can analysts really add any value given their lack of ‘real world’ experience when compared with soldiers/cops/decision-makers who’ve been on the front-line/streets/boardroom for years?) is a serious, if relatively recent, one. But, our troubles don’t end there…

The next question is: “Regardless of how analysts are perceived, does the ‘analysis as art’ methodology (such as it is) produce good results? Are there other methods that produce better results?” If the answers to those questions are ‘yes’ and ‘no’ respectively, than we only have a problem of perception (essentially a community wide Cassandra Complex). Not easy to overcome, but at least there would be some satisfaction in knowing we were getting the right answers.

Unfortunately, we don’t really know the answers to those questions and that’s what Mr. Folker addresses. He argues that a structured approach to intelligence analysis leads to better answers. It’s been awhile since I’ve taken statistics and his sample size is small (unavoidably so, I think) but his results are quite interesting.

It seems to me that as long as structured analytic methods can’t be proven to produce worse results than the intuitive, ‘analysis as art’ methodology they should be incorporated into standard procedure for the law enforcement and military communities. Even if the two methods produce the same results, a structured methodology will give the analysts additional credibility by being able to demonstrate how conclusions were reached and what evidence was used. When dealing with a skeptical audience, hard evidence always beats a wave of the hands and theatrical ‘Voila!’.

I’ve moved away from the ‘analysis as art’ camp over the past couple of years into a middle ground (I might even say logom) that places analysis into a category that must draw upon both intuitive and structured systems in order to produce quality products. We’ll never get to a point where we can make a checklist that people can follow to produce analytical products (and I’m not sure we’d want to even if we could) but, by the same token, we need to get beyond just ‘wingin’ it’ as well.

Advertisements

Posted on 15 July 2011, in Uncategorized and tagged . Bookmark the permalink. 1 Comment.

  1. bloodhoundbetty

    I’ve actually come across many of these same issues in the data forensics world and I would have to say, there’s no simple answer, both fields benefit by a mix of science and art. To me it seems the scientific parts are when five different people can apply the same technique to the same data and get the same answer, the artistic part is where five people can look at the same data and interpret or present it five different, unique ways. For data forensics the science part comes from certain aspects of computer science, for example imaging the bits & bytes of a hard drive. Five examiners image the same drive and will get the same results, or at least better get the same results. The art comes in when it goes to presenting the findings and putting it together with the facts of the case. The same things can be said for analysis, the scientific aspects would be processes like data mining and the artistic aspects would be creating data visualizations of your findings, and the best product would have both.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: