Arms and influence

Political science breakthrough!

Poliblogger's motto is, "Politics is the master science." Well, he won't be surprised to hear that political scientists have discovered a new form of government.

Interrogation that works

This excellent op-ed piece by Stuart Herrington describes how interrogation really works:

In interrogation centers I ran, we called prisoners "guests" and extended military courtesies, such as saluting captured officers. We strove to undermine a prisoner's belief system, which we knew instructed him that Americans are unschooled infidels who would bully him and resort to intimidation, threats and brutality. Patience was essential. We rejected the view that interrogators could merely "take off the gloves" and that information would somehow magically flow if we brutalized our "guests." This notion was uninformed and counterproductive, not to mention illegal, and we made sure our chain of command understood that bowing to such tempting theories would result in bad information.

In other words, you want to "turn" someone, to increase the amount and reliability of information. You don't want to turn them into someone who will lie to you, just to get the excruciating pain to stop. If you doubt Herrington, first go read his book about his experience as a military intelligence professional during the Vietnam War (link to your right).

[Thanks to Misfire! Misfire! Misfire! for the pointer.]

The waste list, part II

[Warning: This post includes a lot of information about modern computer systems. I’ve written it for the average person, but you may not be interested in it at all. It’s important to understand why the US government should not have a useless Terrorist Screening Database (TSDB) four years after the project officially started in 2003, six years after the 9/11 attacks. If you find your eyes glazing over, just skip ahead to the next post in this series. I find this topic interesting, but you may not.]

The obstacles to building the terrorist watch list are the same facing private corporations during mergers and acquisitions. When company A acquires company B, two organizations must now merge incompatible storehouses of information. If company A uses a particular piece of software to manage customer information, and company B uses a different application, the two sets of information won’t match.

Different definitions of "important"
Nowhere is this pain felt more keenly than the customer database—or, more accurately, databases. Without a standard way to describe customers, the two companies’ customer relations management (CRM) systems will have different “data models.” There will be overlap in obvious places: for example, everyone needs the name of a customer. However, the two systems will have many key differences: one CRM application might record detailed information about marketing campaigns; the other CRM application might provide a far sketchier picture of marketing effectiveness.

To complicate matters, no company ever leaves critical applications (CRM, accounting, HR, etc.) untouched. Because these systems manage important information, they need to include the information that’s important to that company. No application ever perfectly describes that information for every possible organization, so some “customization” is necessary to add or modify information. Companies often spend millions of dollars to perform these customizations, so you can see how important they are.

Therefore, mergers and acquisitions immediately create an information management crisis. Day-to-day business operations depend on information that is now stored in incompatible systems. The business and legal costs of leaving the information in this Balkanized form are too high, but the scope of the problem—potentially, millions of records in incompatible databases—is daunting.

Hired guns to the rescue
At this point, corporations look to a combination of software and specialists for help. Many software companies exist purely to provide tools for merging different databases. Each of these software vendors usually has its own specialization, such as reconciling product catalogues, or migrating data from one popular accounting system to another.

However, these software tools are never going to provide a “pushbutton” solution. Company A won’t be able to install the software, click a button, and Voila! the problem disappears.

One obstacle is customization: it’s extremely difficult to preserve the customizations as you move information from system X to system Y, or from both systems into some separate, combined data repository. The software might be able to follow your instructions for handling the added information. However, unless you are dangerously overconfident about the software’s reliability (or, for that matter, your own instructions), you’ll have to double-check the results.

Therefore, another major obstacle to reconciling databases is “quality assurance.” All too frequently, migrated information is garbled, duplicated, or missing. It’s the way of the electronic world, so you’ll need to roll up your sleeves and start digging through a staggering amount of information about sales opportunities, customers, financial transactions, and everything else in these different databases.

To increase the chances of success, while ensuring that the project doesn’t drag on forever, companies hire “systems integrators” (SIs). Most corporations don’t have these kinds of experts permanently on staff, waiting to be called. Better to hire people who handle these projects every day, have a deep understanding of the systems in question, and know the do’s and don’ts that can be learned only through hard experience.

Not all problems are technical
Now that you have the software and specialists in place, these projects may still fail, not for any technical reason. The corporation itself may sabotage the project. Without clear project goals—what data must be preserved, what might be discarded to simplify matters, how quickly does the project need to be done, etc.—expensive systems integration projects might grind on for months or years without completion.

The people within the corporation who really understand the data models might be reluctant to add to the other work they need to perform their participation in the messy, difficult, and lengthy quality assurance process. For example, only the sales managers really know what some of the information about business opportunities really means, so the project can’t succeed without their help. Unfortunately, the sales managers are usually backing towards the door, unwilling to get so tied up in this project that they can’t do their day jobs, making money for the company.

When these problems arise, you’ll inevitably hear a favorite corporate buzzword, “ownership.” Who owns this project? Who owns the data model? Who owns the reports that need to be generated, once the information is finally available?

“Ownership” has two facets, assigning and assuming these responsibilities. A high-level decision-maker has to appoint someone to be responsible. If that person’s duties and authority aren’t clear, there’s no real “ownership” possible, and the project will fail. If you pick the wrong person to assume responsibility, the project will fail. If the importance and urgency of the project isn’t clear to everyone, and not phrased in ways that will motivate them, the project will fail.

When companies hire specialists to handle these projects, they’re often looking for people who can advise them how to handle these all-too-human failings. However, even the best advice will go unheeded if there isn’t a person with real clout—usually someone high in the organization chart—keeping a careful eye on the project.

“Hard” does not mean “impossible”
These projects might sound Herculean, but they’re hardly impossible. The work may take months or even years, but it does get done in hundreds of corporations every year. Feeling the competition snapping at their heels, companies find a way to make these projects successful.

Therefore, you might expect the US government, facing the terrorist threat, would also find a way to make the Terrorist Screening Database (TSDB) and its by-product, the terrorist watch list, successful. A TSDB with 755,000 entries is, by definition, a failure, since its purpose is to identify and track the members of small but dangerous terrorist cells. So what happened?

Next: Some possible reasons for the TSDB’s failure.

Discovering Japan

I'm on a business trip for the next few days, so I'll be posting sporadically. I promise to return with souvenirs for everyone.

Thanks for the gift, but...

An important rule of counterinsurgency is, Gifts that the recipient does not want, or does not need, will never win "hearts and minds."

Case in point: a $38 million accounting system for the Iraqi government, which for the last year, no one in the Finance Ministry is using. Obviously, the Finance Ministry needs accounting. However, it wasn't necessarily ready for state-of-the-art accounting software. Or, perhaps, the problem was the fashion in which American contractors and officials delivered this system, which failed to convince Finance Ministry's employees that this particular system would work for them.

Either way, $38 million is a lot of money to waste learning a basic lesson about "hearts and minds."

The missing WMD report

Armchair Generalist asks an important question: Now that the Democrats control Congress, where's the second half of the WMD report?

Friday follies: Halloween and Lovecraft

Since we're rapidly approaching Halloween, I thought I'd say a few words about H.P. Lovecraft and the Cthulhu mythos. Whatever you think of Lovecraft as a writer, you have to give him credit for two major accomplishments. First, he conceived of horror fiction in an entirely original way. Instead of ghosts, werewolves, and vampires, Lovecraft's stories featured antagonists that resembled nothing from Western folklore. They inhabited a world somewhere between horror and science fiction, in which god-like beings plot the destruction of the world when "the stars are right," while in Antarctica, biologically-engineered creatures that killed their alien creators still lurk.

By cutting free of the conventions of horror fiction, Lovecraft frightens his readers at a more cerebral level. The monsters may be horrible to look at (often indescribably so), but it's the back story that really sticks it to you. The monsters you see are just the front-line troops of a cosmic conspiracy in which the odds are stacked heavily against human beings. Yikes!

Lovecraft also spawned an entire sub-genre of horror fiction. Dozens of writers have contributed to the "Mythos." It's a hoot to see how the more successful authors have kept the Mythos fresh and interesting.
For Lovecraft newbies and veterans alike, you may enjoy the following videos. The first is an interview with Neil Gaiman, in which he describes the lure of things Lovecraftian. The other is the only movie that ever "got" the Mythos. As Gaiman explains, it's hard to successfully film the Mythos, but this five minute flick comes awfully darn close.

The waste list, part I

The Department of Homeland Security’s terrorist watch list is practically useless. Now bulging at 755,000 names, 20,000 new records names are added to the Terrorist Screening Database (TSDB) each month. With the 1,000,000 mark close at hand, the watch list is now unreliable.

The signal-to-noise ratio is enormous, since terrorist organizations are very small. Al Qaeda in 2001 had only 200 members. Larger terrorist organizations, such as Hezbollah, may only have a few thousand “fighters,” many of whom are soldiers and militia, not terrorists. Just so that we don’t forget the threat of domestic terrorism, only a handful of people were responsible for the horrific Oklahoma City bombing, the most lethal act of terrorism in the United States before the 9/11 attacks. Tragically, for every new Mohammed Atta on the list, there are hundreds of people who should not be included at all, and only confuse efforts to deal with any real terrorist threats.

The problems with the list are infuriating on many levels, not least of which is that they are preventable. The challenges of building the watch list are the same “information management” problems that private corporations face every day. Through a combination  of widely-available software, technical veterans who have experience solving the problems, and a generous amount of leadership and urgency, corporations clean up their databases of “mission critical” information far more quickly and successfully. Since the terrorist watch list is at least as important as the customer data for AT&T and Cingular, the American public should be demanding an answer to the question, “What the hell happened?”

Making a list
The terrorist watch list is, to use a tired but accurate phrase, “the first line of defense” in counterterrorism. The Atta cell succeeded because its opponent, the US government, failed to put together a complete picture of its activities (or, in some cases, even its existence). Those who did have key pieces of information either did not recognize their significance, or lacked the ability to get the right people to act on it.

To avoid future terrorist attacks, the watch list is the critical product of knowing who are terrorist leaders and operatives, where they are, and what they are doing. To avoid another 9/11, we don’t necessarily need to broaden the federal government’s license to eavesdrop (which can create a different form of information overload), or invade any more countries. We do need key decision-makers to recognize that this person who learned how to fly a plane, but not how to land, has a real connection to that person, who belongs to an organization that tried to crash an airliner into the Eiffel Tower.

Checking it twice
The watch list is, obviously, stuffed with “false positives,” people who should not have been identified as terrorists. They’re on the list for a variety of reasons: they have names similar to real terrorists; they have some glancing social, professional, or familial connection to a suspected terrorist; someone falsely accused them of being a member of a terrorist organization, or “linked” to it in some vague fashion. These errors feed off each other: people connected to a person falsely accused of being a terrorist fall under the same cloud of suspicion.

Tragically, the list has also omitted people who should have been on it. During the tedious review of just 2,686 records, workers from the FBI’s new Terrorist Screening Center found eight people who should have been flagged as high risk. (They also found 2,118 who should never have been on the list in the first place.) Twenty terrorists were missing from a key report generated for the decision-makers who need to act immediately on any possible terrorist threat. The risk of “omission by confusion” is high, since the decision-makers who receive the list are naturally skeptical of it.

Who’s naughty or nice?
So why is the list a mess? One of the main problems is the novelty of having a single list.
The 9/11 attacks demonstrated, in the worst possible way, that the era of data Balkanization, in which every government agency maintained separate information needed to combat terrorism, had to come to an end. The FAA, FBI, NSA, and other agencies might still have their separate databases, which were needed for more than just identifying and tracking suspected terrorists. At a minimum, these agencies needed to contribute their pieces of the counterterrorism puzzle to someone who would assemble it, and make sure that the right people were looking at it.

That responsibility fell to the Justice Department. In 2003, George W. Bush directed Attorney General to “establish an organization to consolidate the Government's approach to terrorism screening and provide for the appropriate and lawful use of Terrorist Information in screening processes.” Confusingly, the Department of Homeland Security also had a role, to “develop guidelines to govern the use of such information to support State, local, territorial, and tribal screening processes, and private sector screening processes that have a substantial bearing on homeland security.”

Something obviously went very, very wrong with this arrangement. Six years after the 9/11 attacks, the US government has not built a single, reliable database of suspected terrorists. While we spend billions of dollars each month in Iraq, the real “first line of defense” has collapsed.

NEXT: How private corporations handle information management challenges faster and better than the US government.

When you have real alliances...

...You can actually get NATO to escalate the effort in Afghanistan, even though the war isn't terribly popular in Germany and other parts of Europe. (Some good news, at least.)

Hard to argue with him

Turkey's prime minister, Recep Tayyip, doesn't find it hard to justify military strikes against the PKK in Iraq:

"The ball is in our court now, and we will have to do what is necessary on our own if those who have the responsibility do not take action," Erdogan said in Bucharest, Romania's capital, during a joint news conference with Romanian Prime Minister Calin Popescu Tariceanu.

That's the diplomatic way of saying, "Watching the PKK slip over the Iraqi border after killing 17 of our soldiers was the last straw. If the US and Iraq won't deal with the PKK, we will."

Given the level of provocation--years of PKK terror attacks into Turkey--the Turkish military response is restrained. At a time when the Turks might be out for blood, the Turkish military has, so far, only used artillery and air strikes, and very limited ground forces, against suspected PKK positions. Turkey could have gone a lot farther in trying to deal with the PKK--and still might, if the US and Iraq fail to respond.