Numbers matter

Since counterterrorism supposedly leaped to the top of the list of US national security priorities, we've seen very few efforts to measure how much better the US government is doing at stopping terrorist attacks. Americans have been curiously undemanding on this point. Even among the critics of the Bush Administration's counterterrorism strategy, there are few people who have asked for a detailed measures of actual results.

That's what makes the recent report from the Inspector General of the US Department of Justice a bombshell. Perhaps it didn't explode into as many headlines as it should have, or as loudly. However, it is one of the most scathing critiques of the Bush Administration's "everything has changed, get the hell out of our way, we're taking care of it" approach.

Realistically, even the most rigorous efforts to measure the effects of public policy will have small mistakes. However, the errors in the DOJ's counterterrorism statistics are inexcusably large. They also appear to support the contention that, rhetoric aside, the Bush Administration has not taken counterterrorism all that seriously.

Too many errors
It's one thing to get a few statistics arong. It's quite another to have  the majority of numbers wrong, in every category measured. Here's the table from the IG's report that summarizes the frequency of error.

The punchline is fairly obvious. The Department of Justice, including agencies like the FBI, is supposed to be acting as one of the "shields" that protects the United States. (Meanwhile, the "sword" is working overseas.) Within the DOJ, it's apparently OK to be ignorant of how well "the shield" is working.

Unjustified claims of progress
The DOJ regularly claims successes that don't really exist. The amount of statistical bombast is often alarmingly high. For example, only 25 out of 81 alleged terrorist convictions in 2003 were justified. In 2004, the pattern repeats, with only 35 out of 91 convictions justified.

Examples of statistical shenanigans cited in the IG's report include a credit card scammer who had not ties to terrorism, but wound up in the counterterrorist "win" column of convictions. Clearly, we're not talking about a "liner," a case that you could argue did or did not belong on the list. It's the sort of inclusion that smacks of aggressive padding of the estimates, in the hopes that no one would notice.

Hiding mistakes
Some measures that need to be adjusted upwards, according to the Inspector General's report, include the actual number of arrests in terrorism cases. While the DOJ reported only 113 arrests, from 2001 to 2005, of people accused of providing material support to terrorism, the IG found 161 actual arrests.

Undoubtedly, many of the underreported arrests are related to the errors in the number of prosecutions and convictions. Many of the arrests did not lead anywhere. As shown in other studies, many agencies  the US government at large--the DoD, DOJ, CIA, and others--have thrown out a very large net, without regard to who gets caught in it. Valuable time, the good will of people whose cooperation is needed to catch real terrorists, and manpower has been wasted on investigating and arresting the wrong people.

How could this happen?
One of the great challenges in any organization is making something a priority without motivating people to do the wrong things. Just as the body count in Vietnam incented US officers to report unjustified kills, the pressures on the DOJ are undoubtedly leading to unjustified claims of progress.

These distortions are especially bad when a public policy campaign is phrased as a war. While Al Qaeda might consider itself at war with the United States, the mechanisms for preventing future Al Qaeda attacks are not "warfighting." Instead, they more closely resemble organized crime prosections, a long, difficult process of investigating and dismantling a small, secretive, violent organization.

Instead of calming fears and dampening panic within the executive branch, the White House has encouraged these attitudes. Another attack may be imminent, so drastic measures are needed. Better to embrace the mindset of the "1% rule" than approach domestic counterterrorism soberly. As a result, we're left with grave doubts about what threats really exist, and how well US officials are handling them.

But don't take my word for it. Read the report. It's an important document, well worth the time. The IG's methods were careful and convincing--which, unfortunately, you cannot say about many of the people it depicts.