Our Secret American Security State

What is the American intelligence bureaucracy good for? The question is difficult to ask in a serious way in Washington because it risks raising the hackles of career intelligence professionals and their political sponsors at a time when spy agencies remain under pressure to combat resilient if diminished international terrorist groups and to monitor and check Iran’s nuclear program, among other challenges. Yet a serious, transparent review of the intelligence system’s strengths and limitations is overdue.

The past decade has witnessed one of the most egregious misuses of intelligence in American history—the Bush administration’s distortion of information about Saddam Hussein’s terrorist ties and unconventional weapons, in order to justify the invasion of Iraq. It has also seen a surge of paramilitary activity and covert action that has included the operation of secret prisons, the use of torture, and targeted killing. The Obama administration ended officially sanctioned torture, but it has refused to allow official inquiries into how it occurred, and the administration has increased the number of covert, unacknowledged targeted killings through the use of armed, unmanned aerial drones in Afghanistan, Pakistan, Yemen, and elsewhere.

In all, a president who might have challenged the American intelligence bureaucracy and given it a new direction has instead maintained and even expanded what he inherited. Nor has Congress reviewed the hasty organizational reforms it enacted after September 11 or reckoned in depth with the problems exposed by the Iraq disaster. The vital questions that seemed to be begged after the Bush era—about the intelligence system’s scope, effectiveness, costs, outsourcing, legal justifications, and vulnerability to politicization—have remained largely unaddressed.

Intelligence collection is an aspect of self-defense. Some of the work can be distasteful, but there is hardly a kingdom, empire, or democracy in history that has not engaged spies to provide warning about potential surprise attacks, assess the intentions of hostile governments, and ferret out the espionage of other powers. The United States did so in a modest way during the first century and a half of its existence. In an age of slow-moving ships, two oceans rendered strategic surprise unlikely. Pearl Harbor ended that era. By the early 1950s the American intelligence system was swelling into a labyrinth whose size would become much greater than anything imagined by the authors of the Constitution—intelligence became a sprawling, largely self-managing arm of cold war strategies and Pax Americana. The shape of the present intelligence bureaucracy—and the recurring doubts about its accountability to Congress and the public—are traceable to this postwar period.

The mission of cold war spies was to give warning about—and to help prevent—a Soviet nuclear strike on the United States or a conventional invasion of Western Europe, which might escalate into Armageddon. Technology emerged as a relative strength of American intelligence during this period. Spy planes, satellites, and eavesdropping nets provided continuous watch of Soviet missile silos, submarine pens, and fielded armies. Occasionally, the watchers missed critical developments, such as the Soviet deployment of nuclear missiles to Cuba in 1962, but on the whole the system was reliable, and it eventually provided monitoring support for a nuclear test ban, other arms control treaties, and détente.

The intelligence bureaucracy has also expanded into war-fighting as one postwar president after another authorized covert action in direct and proxy conflicts against Soviet and Chinese interests, from Korea to Vietnam to Afghanistan and Nicaragua. These programs drew the CIA into a succession of dirty wars. In what turned out to be the most costly error of all, in Afghanistan, a covert program to support Islamist groups fighting the Soviet occupation helped to foster al-Qaeda.

Intelligence budgets shrank temporarily after the Soviet Union’s collapse, but before any deep structural or organizational changes in the system could take hold, the September 11 attacks reopened Congress’s checkbook. The American security state today is an unwieldy hybrid blending the technology-heavy systems of the cold war period with more recent adaptations intended to address the very different, needle-in-a-haystack problem of tracing and disrupting stateless terrorists. As a proportion of the economy or of federal spending, the bureaucracy is not as large today as it was at the cold war’s height. It remains, nonetheless, enormous and in many ways impenetrable by outsiders.

In their important investigation, Top Secret America, the journalist Dana Priest and the national security analyst William M. Arkin cite, as of 2010, acknowledged annual expenditures of $80.1 billion on national intelligence. But that figure does not include all of the military’s intelligence spending or the Department of Homeland Security’s $58 billion annual budget. Priest and Arkin present striking information about the current system’s reach. Between 2006 and 2010, Arkin constructed a database that documented 182,000 job openings requiring a Top Secret clearance, the highest baseline level of classification. (Above Top Secret there are “compartmented” clearances required for access to particular programs or information such as the names of confidential sources.) According to the authors the government spends $10 billion a year just to manage the secrecy of its secrets.

Top Secret America’s most troubling finding involves the spread of highly compartmented intelligence activities known as Special Access Programs (SAPs). These are often covert actions and can range from unconventional intelligence collection to sabotage and paramilitary violence. The Defense Department is required to transmit a list of many but not all of its SAPs to Congress annually; as of 2010, the list ran to three hundred pages. Priest and Arkin quote John Rizzo, a former CIA acting general counsel, as saying that the number of covert operations launched since September 11 far exceeds the number that were underway during even the busiest cold war periods. James R. Clapper, currently the director of national intelligence, tells the authors, “There’s only one entity in the universe that has visibility on all SAPs—that’s God.” In view of its source, this is a disturbing statement, however facetiously it was intended.

One of the many strengths of Top Secret America is that Priest and Arkin take nothing for granted. They ask basic, even faux-naive questions about the purpose, accountability, and effectiveness of the acronym soup of covert programs, companies, and Pentagon commands created or expanded after September 11. Their analysis is neither naive about the threat posed by al-Qaeda and similar groups, nor credulous about the generals, spies, and bureaucrats who have so dramatically expanded the country’s defenses in response to September 11.

The book unfolds as a kind of detective story. Arkin works from a converted barn in Vermont, where he mines open sources: government websites, help wanted advertisements, requests for proposals issued to contractors, newsletters, and replies to letters he dispatches to federal officials. His work is that of a mapmaker—he accumulates fragments of evidence about new or undisclosed intelligence and secret defense programs, connects diverse sources, and gradually fills in an overarching sketch. Arkin concentrated his research only on Top Secret–level programs. To map all the information about Secret and other less classified data sets would be virtually impossible—there is, he found, just too much of it. At various times during his multiyear investigation, for example, Arkin discovered as many as 15,000 listings at one time for jobs requiring a Top Secret clearance.

Collating and correlating, he handed off promising paths of inquiry to Priest, who knocked on doors and interviewed sources. Priest is a Washington Post reporter who won Pulitzer Prizes for her investigations exposing the Central Intelligence Agency’s secret prison system and the appalling conditions endured by wounded veterans from Iraq and Afghanistan at the Department of Defense’s Walter Reid Army Medical Center. She is perhaps the most dogged and fiercely independent national security reporter of her generation. Many of the most entertaining passages in the book describe her driving around the Washington area while using maps on her cell phone to locate and investigate offices and companies; she wandered through anonymous office buildings, matching names on doors with those in Arkin’s databases. Between them, the authors record, they conducted several hundred interviews and visited at least one hundred sites where Top Secret work takes place.

Arkin and Priest document three important failings of the country’s current intelligence apparatus. The growth of intelligence programs since September 11 has been very poorly managed, producing large amounts of waste and duplication. The system’s inefficiencies are dangerous, because they increase the probability of unintended errors. The system’s scale, finally, has become self-perpetuating. Politicians in both major parties are too cautious and too focused on electoral advantage to assume the political risks of an honest reassessment.

The authors often present their inventory of the system’s weaknesses through on-the-record statements by leading participants. John R. Vines, a retired lieutenant general assigned by former defense secretary Robert Gates to review the Pentagon’s intelligence programs, tells Priest and Arkin that “the complexity of this system defies description” and that it was impossible to tell whether the country was any safer because of all the spending authorized after the September 11 attacks. And yet the retired Marine General James Jones, President Obama’s first national security adviser, conceded that there is no political incentive to confront the problems: “Who wants to be the guy that says we don’t need this anymore and then three weeks later something happens?” He adds, “I don’t think you can ever get it back” to a manageable size.

Profiteering also explains the system’s intractability. In a chapter entitled “The Business Card,” Priest and Arkin portray the intelligence-industrial complex, a web, as of 2010, of about 1,900 companies working on Top Secret contracts. Of the 854,000 Americans with Top Secret clearances, Priest and Arkin estimate that more than a quarter—about 265,000—worked in 2010 not in the government but at for-profit corporations. Private-sector companies have long participated in American defense, but the numbers Priest and Arkin unearthed reflect a speeded-up drive for privatization that has gone haywire. The expansion was supposed to save money. In fact, according to a 2008 study by the Office of the Director of National Intelligence, contractors made up 29 percent of the workforce in the intelligence system but were responsible for about half of the personnel costs. Gates, himself a former career intelligence officer, is quoted saying that contractors cost the Pentagon 25 percent more than federal employees.

And yet the waste is sustained by a revolving-door system in which intelligence officials who make policy decisions about privatization often leave office to join the for-profit companies they have aided. Priest and Arkin cite the case of J. Michael McConnell, a former director of national intelligence who has twice rotated between government intelligence posts and executive positions at the contractor Booz Allen. As the country’s intelligence chief during the second term of President George W. Bush, McConnell “was a strong advocate for increasing the contracting work of intelligence companies,” the authors write. When McConnell left the Bush administration, he returned to Booz Allen to run the firm’s national security contracting work; his total annual compensation was $4.1 million.

Private companies may usefully contribute to intelligence, particularly in the creation of new technologies. What Priest and Arkin describe, however, often involves the outsourcing of core functions such as intelligence collection, analysis, and covert action. Even if this policy were not financially wasteful, it would be folly. Intelligence work by its nature often confronts the practitioner with moral compromises, in which the extent of a particular compromise must be weighed against a larger mission to protect the public. To add a profit motive to such work is to assure a rise in corruption and misjudgment.

The findings presented by Priest and Arkin are troubling, but they at least suggest possibilities for improvement, for example much stricter limits on the use of private intelligence. The conclusions drawn by Paul R. Pillar, who served for twenty-eight years at the CIA and the National Intelligence Council, are more dispiriting. Pillar argues that reform is a foolish fantasy—indeed, that the perpetual search for reform of American intelligence is one of the system’s main problems. Intelligence and US Foreign Policy is his third book, and it is a fascinating, important, and peculiar volume.

The book essentially has four parts. The first is a brilliant account of the role of intelligence in the run-up to the invasion of Iraq in 2003. The second is a broader, provocative review of the uses of intelligence—or the absence of such uses—in American foreign policy decision-making, especially at critical moments in postwar history. Pillar then presents a revisionist, defensive, and ultimately unsatisfying account of intelligence and the September 11 attacks. This culminates in an ardent, elaborate denigration of the 9/11 Commission cochaired by former New Jersey governor Thomas Kean and former congressman Lee Hamilton, and particularly its staff director, the historian Philip Zelikow. He argues that the 9/11 Commission used a “badly flawed and highly inaccurate account” of the CIA’s work on international terrorism before September 11 in order to advance a preconceived plan to reorganize the intelligence agencies. He points out correctly that the commission did not credit as fully as it might have some of the prescient analysis about al-Qaeda and its affiliates produced by Pillar and his colleagues during the 1990s. He also points out accurately that the commission’s bipartisan membership was not as critical as it should have been of the Bush administration’s sluggish reaction to acute CIA warnings about al-Qaeda delivered throughout the first eight months of 2001.

Yet his case overall is selective and overly protective of his former employer. He does not address, for example, the fact that the commission’s eventual best-selling narrative of the September 11 attacks was based in part on information supplied by the CIA, which was obtained from Khalid Sheikh Mohammed during interrogations that employed what the Red Cross later described as torture. American courts do not admit evidence obtained under torture not only because it would be unethical but also because such testimony is often unreliable. Pillar offers no sense of whether the torture or the commission’s use of information obtained by torture bothered him. Finally, he reflects on the impulse of outsiders to reform the intelligence bureaucracy, which he sees as delusional.

Pillar’s most important argument is that “the overall influence—for good or for ill—of intelligence on major decisions and departures in US foreign policy has been negligible…. Policy has shaped intelligence more than vice versa.” To make this case, Pillar reviews the Korean War, the Cuban missile crisis, Vietnam, Henry Kissinger’s “back channel” opening to China, the Indo-Pakistani war of 1971, President Jimmy Carter’s response to the 1979 invasion of Afghanistan, and President Ronald Reagan’s policies toward the Soviet Union.

In all of these instances, Pillar argues, the American intelligence community “had almost no guiding role in great decisions.” Analysis derived from secret information “has played an important supporting role” and it “also has had important things to say on the subjects about which the great decisions were made.” But it did not influence those decisions very much. Domestic politics, the assumptions and prejudices about global affairs held by presidents and their advisers (such as the “domino theory” in Southeast Asia during the Vietnam War), and gut instincts have been far more important than reflections upon considered intelligence assessments delivered by the bureaucracy.

In some of the cases Pillar reviews, such as the Vietnam War, he argues that the intelligence analysis available was largely accurate but presidents and their advisers paid it little heed. At other times, when the intelligence was wrong, that didn’t matter, because it too was ignored. The CIA persistently underestimated Mikhail Gorbachev’s willingness to carry out radical reforms, for instance, but Reagan and Secretary of State George Schultz discounted the assessments, preferring their own, more optimistic judgments.

Rather than being influenced by intelligence analysis, Pillar writes, it is more common for presidents and their advisers to subtly shape and distort intelligence reporting through pressure and “politicization.” This provides an underpinning of Pillar’s excellent account of the misuse of intelligence to justify the invasion of Iraq. The story has been told before of how Bush and his advisers decided almost immediately after September 11 to attack Iraq and then used intelligence to sell the war without acknowledging to the public or Congress that the decision to invade had already been made, but Pillar provides a telling and comprehensive new perspective from the inside. After 2000, he was the senior officer on the National Intelligence Council, the country’s leading institution for all-source intelligence analysis. He was responsible for political and economic assessments of the Middle East. He concludes from his direct experience that the Bush administration did not value intelligence about Iraq before the war except as marketing material to justify an invasion already decided upon: “No Bush administration policymaker asked my office for any classified assessment on anything having to do with Iraq until a year into the war.”

While defending his own work as an intelligence analyst, Pillar settles occasionally into a defensive crouch. In the Iraq section of his book, the problem is incidental. In his treatment of the CIA’s role in warning about the September 11 attacks, however, it is self-defeating. Pillar has a straightforward and defensible case to make. Before September 11, he and others at the CIA provided repeated, loud, and clear strategic warnings about al-Qaeda’s intent to cause mass casualties by attacks on the US. Policymakers, particularly in the Bush administration, took inadequate notice. For all of their urgent strategic warning, however, the intelligence agencies failed to provide any specific, tactical warning about the September 11 conspiracy. Pillar glosses over the failures of CIA analysts to recognize and share information they possessed about two of the hijackers who had entered the United States in 2000—information that, if seized upon and exploited earlier, might have prevented the attacks.

Pillar acknowledges these “specific and unmistakable errors” but does not examine them. In general, he regards harping about failures of tactical warning by the intelligence community as misguided and unrealistic, part of a “spectator sport” played by Congress, journalists, and other armchair spies who fail to appreciate the innate imperfectability of intelligence. He writes:

We should not be surprised to be surprised—and I refer here to tactical surprise, which is harder to reduce, let alone eliminate, than strategic surprise, chiefly because it involves unobservable and perhaps unattainable things such as an adversary’s secret plans.

Yet earlier, two of the September 11 attackers had been observed at a meeting with a dangerous al-Qaeda leader in Malaysia, and the CIA knew they had visas to enter the United States. Agency analysts looked at this reporting but failed to see its significance. Even if we stipulate that the mistakes made by these analysts were owing to overwork and human imperfection, they nonetheless deserve auditing.

Pillar’s insistent modesty about what intelligence can do well is not confined to the subject of tactical warning. Intelligence can “help to manage the uncertainty” inherent in global affairs, he writes. It can also “distinguish true uncertainty from simple ignorance,” and it can “identify possible outcomes” in foreign and defense scenarios “and perhaps give a sense of the relative likelihood of each one.” But that is about it. For this we are expending $81 billion or more a year. Would not a few magazine and newspaper subscriptions suffice?

During the decade after September 11, the overriding mission transmitted by the White House to the intelligence community has been to prevent a major terrorist attack against the United States and to degrade or defeat al-Qaeda. No major attack has occurred and al-Qaeda today is greatly reduced; its longtime leader, Osama bin Laden, is dead because of an American intelligence breakthrough. Counterstrike, by the New York Times national security reporters Eric Schmitt and Thom Shanker, is a carefully reported, briskly paced account of the counterterrorism campaigns that followed the September 11 attacks, and it provides a reliable sense of what has gone right, without cheerleading or obfuscating about failures. The book is particularly strong on the period since 2007, including the stepped-up effort to dismantle al-Qaeda’s core leadership along the Afghanistan–Pakistan border. Schmitt and Shanker bring forward new details about how these intelligence-led operations have worked despite the breakdown in trust between the CIA and the principal Pakistani intelligence service, the Inter-Services Intelligence Directorate.

Counterstrike argues that as the campaign against al-Qaeda has evolved, American strategists in the intelligence system and at the Pentagon have returned to cold war–era thinking about deterrence derived from the lessons of Soviet containment. Both as strategic analysis and historical analogy this comparison to nuclear deterrence is unconvincing. Al-Qaeda is hardly the Soviet Union. It is reasonable to fear the group’s historical interest in acquiring weapons of mass destruction but it is tendentious to equate that threat to the one posed by the Soviet Rocket Forces, which for decades had thousands of actual atomic weapons on hair-trigger alert, pointed at American cities. The notion that we should defend against terrorism by drawing upon the precedents of anti-Soviet deterrence makes sense only if the impulse is understood perversely—as an indicator of how the intelligence and defense bureaucracies rationalize their outsized programs by mustering politically safe precedents for budget writers in Congress.

As Priest and Arkin write, one reason the intelligence system remains so bloated, inefficient, and difficult to oversee a decade after September 11 is because “the government has still not engaged the American people in an honest conversation about terrorism and the appropriate US response to it,” which would involve reappraising al-Qaeda’s strengths and weaknesses more forthrightly, closing “the decade-long chapter of fear” of the unknown in terrorism, and confronting “the colossal sum of money that could have been saved or better spent.”

After September 11, newspaper Op-Ed pages were full of recommendations for radical departures in American intelligence, changes that might place new emphasis on lean and adaptable operations. There was much talk of a long-term development of “human sources of information”; of the need for risk-taking and the bold penetration of what are known in the intelligence agencies as “denied areas,” such as Iran and North Korea. Some of that ambition has been fulfilled; it is difficult to measure how much, since so much of the detail of post–September 11 covert action and intelligence collection remains secret.

What is plain, nonetheless, is that the larger story of the American intelligence system is one of continuity. The bureaucracy has defended itself from outside investigation and oversight and has followed many of the trajectories set during the Eisenhower years. The relative strengths of tactical American intelligence tradecraft today include innovative technology, vacuum cleaner–like collection of electronic data worldwide, computer algorithms that sort valuable information from noise, and the bludgeoning effects on adversaries of huge if wasteful spending. These methods look very similar to those of the anti-Soviet intelligence system. The bureaucracy’s weaknesses—inefficiency, ignorance of local cultures, revolving doors, self-perpetuation, vulnerability to political pressure, and an overall lack of accountability—are deeply familiar, too.