A Prisoner of His Own Restraint

Over the course of American history, the US Supreme Court has usually been the most conservative of the three branches of government, often to a reactionary extreme. Indeed, an objective observer might well conclude that it has frequently encouraged our nation’s darkest tendencies.

In the decades immediately preceding the Civil War, the Court not only rigidly enforced slave laws but also declared that even free African Americans were not US citizens (Dred Scott v. Sandford, 1857). It reduced Native Americans from sovereign nations to wards of the state (Cherokee Nation v. Georgia, 1831), which was free to remove them from their treaty-prescribed lands and send them on the deadly Trail of Tears to bleak reservations hundreds of miles west. In the decades following the Civil War, it undermined Reconstruction (the Civil Rights Cases of 1883), legitimized Jim Crow (Plessy v. Ferguson, 1896), denied women the right to vote (Minor v. Happersett, 1875), and treated union activities as illegal conspiracies (Loewe v. Lawlor, 1908).

In the first decades of the twentieth century, the Court went to great lengths to hold progressive legislation unconstitutional (for example, in Lochner v. New York, 1905), especially laws that sought to protect women and children from exploitation (Adkins v. Children’s Hospital, 1923). And when, in response to pressure from (and ultimately appointments by) President Franklin Roosevelt, the Court somewhat relaxed its disapproval of federal social welfare legislation, it continued in many cases to show a callous disregard for the rights of individuals and minorities, as demonstrated by such decisions as its cavalier approval of the internment of loyal American citizens of Japanese descent during World War II (Korematsu v. United States, 1944).

There was a brief period in the 1950s and 1960s when the Court sought to recognize its past mistakes and move forward, and while the advent of the Warren Court in 1953 was something of a fluke (no one remotely expected that Earl Warren and William Brennan—both nominated by a Republican president, Dwight Eisenhower—would develop into the great champions of liberty they became), in its relatively short duration it demonstrated how the Court could, if it wished, be a leader in the enhancement of civil rights and the protection of minorities. But beginning in the 1970s, the Court gradually resumed its traditional conservative stance, for example by limiting the Warren Court’s efforts to combat police misconduct (City of Los Angeles v. Lyons, 1983). And now, thanks to President Donald Trump’s appointments, the current majority has reduced some of the Court’s most progressive past decisions to ashes.

What accounts for the Court’s being so often on the wrong side of progress and history? Democratic Justice, an impressive new biography of Felix Frankfurter by Brad Snyder, a professor at the Georgetown University Law Center, gives us a chance to consider that question. Though as a lawyer and academic he was a great liberal, as a justice Frankfurter increasingly facilitated the Court’s conservative tendencies. Snyder, who greatly admires him, contends that his principles, formed early in his adult life, remained consistent throughout his career. But a more detached observer may well view it as a foolish consistency.

Frankfurter was born in Vienna in 1882 to parents of modest means but high hopes who immigrated to the United States when he was twelve. Although he spoke no English when he arrived, a dozen years later he graduated from Harvard Law School with the highest grades in the class. He then sought employment in the established law firms of New York, but despite his grades, his obvious brilliance, his prodigious energy, and his buoyant personality, he was rejected by almost all of these “white shoe” firms because he was a Jew.

It may have been for the best. Even though, thanks to the strong endorsement of the dean of Harvard Law School, James Barr Ames, Frankfurter finally found a position at a New York firm, he soon realized that the stuffy confines of the Wall Street legal world were not for him, so he accepted an unsolicited offer to be an assistant US attorney in the Southern District of New York. In Snyder’s words, the offer “changed his life.”

The offer came from Henry Stimson, whom President Theodore Roosevelt had just appointed US attorney for the Southern District (which includes Wall Street) and directed to prosecute the economically powerful trusts—the monopolies and oligopolies—that he had promised to bring to heel. To accomplish this, Stimson decided to replace the politically appointed assistant US attorneys typical of that period with energetic young lawyers selected on their merits. The very young Frankfurter was his first choice. (In my chambers, I keep a photo of Stimson’s first class of merit hires, in which Frankfurter, though twenty-five years old, looks like he is about fifteen. It also appears in Snyder’s book.)

Frankfurter more than met Stimson’s expectations, successfully prosecuting, for example, Charles Heike, the secretary and treasurer of the American Sugar Refining Company (the so-called Sugar Trust that controlled 98 percent of the US market), for a variety of monopolistic practices that had been allowed to continue for years. Frankfurter first convicted lower-level executives and then squeezed them for information about Heike—a method that was used repeatedly to prosecute high-level executives in the years to come (though it has too often been abandoned in the past two decades).

Frankfurter’s successes (which included winning some important cases on appeal) commended him to Stimson, and they became fast friends. When President William Howard Taft appointed Stimson secretary of war in 1911, he brought Frankfurter to Washington as one of his aides and introduced him to the capital’s power structure. This included, among many others, the “people’s lawyer” Louis Brandeis (not yet on the Court) and the “great dissenter” Justice Oliver Wendell Holmes Jr. Both Brandeis and Holmes, though very different in background and temperament, strenuously disagreed with the Court’s repeated rejection of progressive legislation designed to protect workers.

Frankfurter’s intelligence and gregarious personality soon made him popular with Brandeis and Holmes, and they in turn reinforced his growing belief that the Supreme Court should be far more restrained in its review of legislation and defer to the other branches of government except in extreme circumstances. Frankfurter termed this the “liberal construction of the Constitution”—a somewhat unusual use of the term “liberal,” though one that he adopted because he viewed a narrow interpretation of the Constitution as the best way to enable progressive legislation.

For similar reasons, Frankfurter during this period particularly identified with the progressive wing of the Republican Party led by Theodore Roosevelt. But when, as a result of the falling out between Roosevelt and Taft, Woodrow Wilson was elected president in 1912, he realized his employment in the War Department was in jeopardy. Once more his admirers at Harvard Law School came to his rescue, offering him a faculty appointment that he accepted in 1914. He quickly became one of the most respected and powerful professors there and was eventually named the Byrne Professor of Administrative Law, a position he held until 1939. (It is now jointly held by retired Supreme Court Justice Stephen Breyer and my brother Todd Rakoff.) At the same time Frankfurter continued his involvement in progressive causes and public affairs. Once the United States entered World War I, he took a leave from Harvard to serve as assistant secretary of labor, in which capacity he was deeply involved in negotiating settlements between unions and corporations whose disputes threatened to disrupt much-needed war supplies.

On June 3, 1918, the Supreme Court held in Hammer v. Dagenhart that a child labor act passed by Congress in 1916 prohibiting the interstate shipment of goods made by children under fourteen years of age was unconstitutional. Frankfurter, still in Washington, responded by forcing the inclusion in government war contracts of a prohibition against the use of child labor. But the Hammer decision further reinforced his belief that the Court should exercise much greater restraint in assessing the constitutionality of state and federal legislation.

During this period, Frankfurter—who was unusually adept at friendships with persons of varying backgrounds—formed a bond with Assistant Secretary of the Navy Franklin Roosevelt, although Franklin’s wife, Eleanor, described Frankfurter to Franklin’s mother as “an interesting little man but very Jew.” Nonetheless, it was a bond that had major implications when Roosevelt became president in 1933.

After briefly attending the Paris Peace Conference, as a member not of the US delegation but of the Zionist one (Brandeis had converted him to Zionism), Frankfurter returned to Harvard in the fall of 1919 and was immediately swept up in an effort by members of the Harvard Corporation to rid the university of “Bolshevists,” among them Frankfurter. The movement was part of the nationwide red-baiting activity of this period, including the infamous Palmer Raids organized by J. Edgar Hoover, but one can also see in it parallels to McCarthyism, not to mention more recent efforts to rid academia of professors expressing unpopular views. Despite the claims that Frankfurter’s presence on the faculty discouraged right-thinking alumni from contributing money to Harvard (and what could be more important than that?), he was not fired. One of his prominent supporters was the respected New York judge Benjamin Cardozo, who later that year officiated at Frankfurter’s marriage to Marion Denman, the daughter of a Congregationalist minister.

Meanwhile the Supreme Court, as a result of a series of conservative appointees, was becoming more intolerant of social welfare legislation, using constitutional guarantees of “due process” as a means of guaranteeing unfettered “freedom of contract” for American capitalists and their employees. This precipitated a private exchange between Frankfurter and the federal judge Learned Hand over whether, notwithstanding the Court’s misuse of due process to invalidate such legislation, constitutional due process could properly be a vehicle for protecting minorities and their civil rights. The “possible gain,” Frankfurter wrote to Hand, “isn’t worth the cost of having five men without any reasonable probability that they are qualified for the task, determine the course of social policy for the states and the nation.” In other words, his view was that logical consistency demanded equal restraint by the Supreme Court in both situations.

Frankfurter’s deeply held but perhaps naive belief was that voters would ultimately come around to supporting not only progressive social welfare policies but also the civil rights and civil liberties of unpopular individuals and minorities, and he tried to convince legislators and other government officials to favor such causes. He also actively promoted the careers of two of the very few black students then at Harvard Law School, Charles Hamilton Houston and William Hastie, who, with his help, went on to achieve great success (the latter as a distinguished federal judge).

But Frankfurter’s identification in the public eye as a liberal hero was most solidified by his participation in the defense of Nicola Sacco and Bartolomeo Vanzetti, two anarchists who were charged with a Massachusetts murder on flimsy evidence and despite strong alibis, but who were nonetheless convicted, seemingly because of their radical views. Although Frankfurter was not involved in their trial, he became convinced of their innocence when he learned that although the jury was told that the state’s ballistic expert had concluded that the bullet that killed the victim came from one of the defendants’ guns, he only had evidence that the bullet could have come from any .32-caliber Colt automatic pistol.

Frankfurter then threw himself into the case, not only assisting the defense counsel but also writing several public articles dissecting the weakness of the government’s evidence. (Interestingly, the main response came from another famous law professor, John Henry Wigmore, whose treatise on evidence remains a classic to this day but who detested anarchists.) Despite heroic efforts, however, Sacco and Vanzetti were eventually executed. And Frankfurter faced renewed calls for his firing from Harvard, which he successfully resisted. By the end of the 1920s, he had assumed Brandeis’s mantle as “the people’s lawyer.”

Frankfurter was not blind to the fact that liberal Republicans like himself were then (as they are now) a dying breed, and that the focus on liberalism had shifted to the Democratic Party. But he was also one of the first to recognize that the future leader of the Democrats was not the popular New York governor Al Smith but his old friend from his Washington days, Franklin Roosevelt. Snyder recounts in vivid detail how Frankfurter went from Roosevelt’s friendly supporter to his trusted confidant to, in the words of a somewhat disparaging article in The Saturday Evening Post, “the most influential single individual in the United States.” This was a reflection not only of his frequent meetings with Roosevelt after he became president (it was said that he spent two nights a week at the White House) and his impact on the initiation and passage of such important though controversial laws as the Securities Acts of 1933 and 1934 and the National Labor Relations Act, the Social Security Act, and the Banking Act of 1935, but also his successful placement of many of his former pupils in important positions in the New Deal.

Yet the Supreme Court was making governing difficult for FDR by constantly invalidating his New Deal legislation. In 1935 it held the National Industrial Recovery Act and the Farm Bankruptcy Act unconstitutional. In 1936 it invalidated the federal Guffey Coal Act and New York’s minimum-wage law. In response, Roosevelt, after his resounding reelection in 1936, launched his plan to expand the number of justices from nine to fifteen. This was a step too far for Frankfurter, who privately voiced his opposition. But when Roosevelt asked him to keep his opposition private, he agreed. While this doubtless reflected his loyalty to FDR, in the same phone call Roosevelt told Frankfurter that he planned to put him on the Supreme Court, which may have greased the wheels of persuasion. And the threat of court packing, though widely denounced, appeared to have an effect, for starting in 1937 the Court began upholding New Deal legislation, although by a bare majority of five to four.

Eventually Roosevelt was able to mold the Court more to his liking when, after the retirements of two justices and the death of a third, he appointed Hugo Black, Stanley Reed, and then Frankfurter to replace them. Frankfurter’s appointment was to the “Jewish seat” previously held by Benjamin Cardozo, and as in the case of the Court’s first Jewish justice, Louis Brandeis, his confirmation was anything but smooth. Snyder writes, “Frankfurter experienced the first modern Supreme Court confirmation hearing [in that] people publicly testified against him.”

They accused him of being a Communist, as proven by his support for Sacco and Vanzetti, not to mention his being, in the words of one witness, “a Jew from Austria.” In accusations that call to mind those later made against President Barack Obama, his opponents alleged that Frankfurter, who became an American citizen when his father became one, was never properly naturalized, an allegation that was only refuted when his former pupil Dean Acheson located a copy of the father’s naturalization papers deep in the Treasury Department’s files. Nonetheless, Frankfurter’s nomination was confirmed by a voice vote of the Senate, and on January 30, 1939, he was sworn in as an associate justice of the Supreme Court—the same day that Hitler gave a speech predicting “the annihilation of the Jewish race.”

Liberals were initially thrilled by Frankfurter’s elevation. But they quickly learned that his conception of a “liberal” Court was rather different from theirs. The conflict was even more noticeable within the confines of the Court, where Frankfurter’s belief that progress best lay in limiting its review of state and federal legislation so as to let the democratic political process hold sway repeatedly clashed with the view of its newest member, William O. Douglas, that the Court’s responsibility was to protect the civil rights and civil liberties of minorities and individuals.

In the years when a conservative Court was busy invalidating progressive legislation, the considerable difference between these views had been less apparent, because of the Court’s hypocritical employment of either or both theories depending on its purposes. Thus its invocation of the alleged civil right of “freedom of contract” to invalidate labor protection laws was matched with its defense of “democratic” legislative prerogatives to penalize seditious speech. But the considerable gap between these two theories of the Court’s “natural” role now became more evident as the FDR appointees began to make their presence felt.

The first such case in which Frankfurter wrote the Court’s opinion was Minersville School District v. Gobitis (1940). Two children of Jehovah’s Witnesses had been expelled from their public elementary school for refusing to salute the flag. While their parents argued that this was a violation of their freedom of religion because the Jehovah’s Witnesses held that saluting the flag was an insult to God, the school board argued that saluting the flag was essential to teaching respect for the nation. Although the lower courts had agreed with the parents, a majority of the Supreme Court justices sided with the school board, and the case was assigned to Frankfurter. In words very reflective of his overall judicial philosophy, he wrote that the Court had no business denying the school board “the right to select appropriate means” to foster national unity among children during their “formative period in the development of citizenship.” If the parents did not like what the school board was doing, the solution was not judicial intervention but the democratic political process:

To fight out the wise use of legislative authority in the forum of public opinion and before legislative assemblies rather than to transfer such a contest to the judicial arena, serves to vindicate the self-confidence of a free people.

This was utter nonsense. It could not reasonably be supposed that Jehovah’s Witnesses would find support for their refusal to salute the flag “in the forum of public opinion.” The only possible support for their exercise of their unpopular religious belief had to come from the courts and the Constitution, and in a ringing dissent, Justice Harlan Fiske Stone said as much. But as far as Frankfurter was concerned, it was a simple application of his philosophy of judicial restraint—and a clear portent of where he would stand on similar disputes yet to come.

In another indication that Frankfurter’s approach was different from that of his colleagues, he continued to advise Roosevelt on issues ranging from political strategy to controversies likely to come to the Court in the immediate future. This was ethically indefensible, but Frankfurter was loath to relinquish the considerable influence he wielded as the president’s favorite adviser. In 1940, at his strong urging, Roosevelt, with war raging in Europe, appointed Frankfurter’s mentor, Henry Stimson, once again as secretary of war, a post he held throughout World War II. The columnists Drew Pearson and Robert Allen wrote a couple of years later, “Justice Frankfurter has more to do with guiding our destinies of war than anyone in Washington.”

Meanwhile Frankfurter was increasingly aligning himself with the more conservative members of the Court, even in cases involving civil liberties. In Bridges v. California (1941), which overturned on free speech grounds the criminal contempt conviction imposed by a state judge on labor leader Harry Bridges for threatening to call a strike if a then-pending labor dispute (in which Bridges was not personally involved) went the “wrong” way, Frankfurter wrote a lengthy dissent arguing that the right of free speech had to be balanced against “the right of California to keep its courts free from coercion.” A few years later, he joined the Court’s majority in Korematsu v. United States. Just how far Frankfurter was prepared to carry his notions of judicial restraint was well illustrated by his brief concurrence in Korematsu:

To find that the Constitution does not forbid the military measures now complained of does not carry with it approval of that which Congress and the Executive did. That is their business, not ours.

In other words, the enforcement of even the most basic civil rights of citizens is not the business of the courts, at least according to Frankfurter.

It is important to note that Frankfurter was among the least personally prejudiced of the Court’s members. In 1948 he hired its first African American law clerk, William T. Coleman Jr., who went on to a distinguished career in law and government. But when it came to deciding cases, Frankfurter felt compelled to put aside these “personal preferences” in favor of judicial restraint. That year he wrote the Court’s opinion in Goesaert v. Cleary upholding a Michigan law outlawing women as bartenders. In language more conclusory than reasoned, he asserted that the constitutional right of equal protection of the laws “does not preclude the States from drawing a sharp line between the sexes.” One might well ask: Why not?

In short, as a Supreme Court justice Frankfurter became increasingly the prisoner of his own intense belief in judicial restraint. With the advent of the activist Warren Court, this put him ever more at odds with a majority of his colleagues in the years before his retirement in 1962. For example, he thought the notion that the Constitution could mandate “one man, one vote” and thus require reapportionment in Baker v. Carr (1962) was totally out of line with the Court’s purpose and “dangerous to our whole system”—even though the Equal Protection Clause of the Constitution means little if it does not guarantee that all voters be lawfully treated as equals.

But even though Frankfurter’s belief in judicial restraint became more rigid as time went on, his personal devotion to human progress and equality, which likewise dated back to his earliest days, sometimes prevailed. It is not true that Frankfurter was on the verge of dissenting in Brown v. Board of Education (1954), a canard attributed to his enemy Douglas and supposedly based on Frankfurter’s involvement in persuading the Court to order reargument in Brown before the final decision was reached. As Snyder shows in great detail, Frankfurter did so in order to allow time for the Court to reach a unanimous decision in Brown, which he totally supported.

One of the many virtues of Snyder’s book is the wealth of detailed evidence he provides for each of his assertions. His own assessment of Frankfurter’s virtues and faults is that he was prescient in seeing how the supposed lack of restraint of Warren Court jurisprudence could come back to haunt American liberals once the Court returned to its more traditional conservative stance. My view, however, is that Frankfurter got so carried away with his philosophy of restraint that he failed to recognize basic principles of checks and balances inherent in our constitutional design, and in particular the natural purpose of the Court to protect individuals and minorities against excesses in which elected officials and legislators too often engage.

Moreover, it is easy to pay lip service to a particular judicial philosophy in order to achieve any desired result. For example, the Court’s recent decision in Dobbs v. Jackson Women’s Health Organization, overruling Roe v. Wade, can be interpreted by its supporters as an exercise in Frankfurter-like restraint that leaves the issue of abortion to state legislatures, while its opponents can view it as the Court’s refusal to exercise its inherent responsibility to protect individual rights.

What may not be gainsaid, however, is that the Court has now returned to its historically conservative tendencies with a vengeance that can only be called reactionary. And if you put aside arguments over judicial philosophy and look at the practical results, it is worse than that. In the last few days of its most recent term, the Court released a series of decisions that, whatever their purported rationales, made the world a more dangerous place: more dangerous for poor people of color, who can no longer effectively seek redress for certain forms of police misconduct; more dangerous for women, who in many states must now resort to backroom abortions and face imprisonment for doing so; and more dangerous for Americans generally, who can no longer hope to meaningfully curb the increase in gun violence now plaguing our nation and whose government will find it ever more difficult to alleviate the climate change that imperils our planet. It is hard to believe that Felix Frankfurter would have been comfortable with such results.