J. Edgar Hoover: Building American Counterintelligence

J. Edgar Hoover (1895–1972) ran the Federal Bureau of Investigation and its predecessor, the Bureau of Investigation, for 48 years, from May 1924 until his death in May 1972. In that time the Bureau went from a small investigative office with a poor reputation to the country’s national police service and its lead agency for counterintelligence inside the United States. Much of the infrastructure that American law enforcement and counterintelligence still rely on, from the national fingerprint files to the crime laboratory, was built on his watch.
His tenure also produced the Bureau’s domestic intelligence programs, which a Senate committee documented in 1975 and 1976 and which led Congress to limit the FBI director to a single ten-year term. Both halves of the record are part of the history of American counterintelligence. This page sets out what was built, what the counterintelligence cases were, and what courts and Congress later found.
At a glance
- Born: 1 January 1895, Washington, D.C. Died: 2 May 1972, Washington, D.C.
- Education: George Washington University Law School (LL.B., 1916; LL.M., 1917).
- Justice Department: joined 26 July 1917; head of the General Intelligence Division, 1919; assistant director of the Bureau of Investigation, 1921.
- Director: acting director from 10 May 1924, director by the end of that year, and director until his death. The Bureau became the Federal Bureau of Investigation in 1935.
- Counterintelligence record: the Duquesne spy ring, the Pastorius saboteurs, the Special Intelligence Service in Latin America, the Venona-based Soviet espionage cases, Rudolf Abel, and Operation SOLO.
- Domestic programs: COINTELPRO (1956–1971), documented by the Senate’s Church Committee.
(FBI; FBI timeline)
The Justice Department and the Palmer Raids
Hoover joined the Justice Department in 1917 after night study in law at George Washington University. In 1919 he was put in charge of the department’s new General Intelligence Division, which built an index of suspected radicals. That index supported the mass arrests of November 1919 and January 1920 known as the Palmer Raids, after Attorney General A. Mitchell Palmer (Church Committee, Book III).
The raids were tested in court. In Colyer v. Skeffington, 265 F. 17 (D. Mass. 1920), Judge George W. Anderson ordered the release of detained aliens held for deportation, finding that the proceedings against them had been conducted unlawfully. In 1924 Attorney General Harlan Fiske Stone limited the Bureau to investigating violations of federal law and ended its political intelligence work. The Church Committee later noted that this restriction held for a little over a decade (Church Committee, Book II).
Building the Bureau
Stone named Hoover acting director on 10 May 1924, when he was 29. The Bureau then had about 650 employees, including 441 special agents in nine field offices (FBI, archived history). The innovations of his first decade are well documented:
- Professional standards (1924 onward). Hoover dismissed agents he judged unqualified, ended promotion by seniority, introduced uniform performance appraisals, and set up a system of regular field-office inspections. New agents had to be between 25 and 35, pass background checks, interviews, and physical tests, and have training in law or accounting. A formal training course for new agents followed (FBI, archived history).
- The national fingerprint file (1924). On 1 July 1924 the Bureau opened an Identification Division that consolidated the fingerprint collections of the Leavenworth penitentiary and the International Association of Chiefs of Police, about 810,000 cards in all. It became the country’s central repository and is the ancestor of today’s Criminal Justice Information Services Division (FBI Law Enforcement Bulletin, July 1959; FBI CJIS).
- National crime statistics (1930). On 11 June 1930 Congress authorized the Bureau to collect and compile uniform crime statistics for the whole country, the program now known as Uniform Crime Reporting (FBI timeline).
- The crime laboratory (1932). The Bureau’s Technical Laboratory opened on 24 November 1932 with one examiner, Special Agent Charles Appel, and performed 963 examinations in its first year. It provided forensic work to state and local police as well as the Bureau (FBI Laboratory).
- Police training (1935). The FBI National Academy, first called the Police Training School, opened on 29 July 1935 with 23 officers from around the country. It became the standard senior course for American police leaders and later admitted officers from abroad (FBI).
- Public fugitive notices (1950). The Ten Most Wanted Fugitives program began on 14 March 1950 (FBI).
These were law-enforcement innovations, but they mattered for counterintelligence. A service that can match a fingerprint, examine a document, and train a national network of police has the investigative base that spy cases require.
From crime to counterintelligence
In August 1936 Hoover met President Franklin Roosevelt at the White House. According to Hoover’s own memorandum, which is the only contemporaneous record, the President asked for systematic intelligence on fascist and communist activity in the United States. The Church Committee and the General Accounting Office later found that the authority this meeting produced was vague, and that “subversive activities” was never defined (Church Committee, Book II; GAO, 1976).
On 26 June 1939, as espionage cases multiplied, Roosevelt directed that espionage, counterespionage, and sabotage matters be handled by the FBI, the Army’s Military Intelligence Division, and the Office of Naval Intelligence, working together. The Bureau had investigated 634 espionage cases in 1938, against an earlier average of about 35 a year (Webb, Studies in Intelligence). That directive made the FBI the country’s domestic counterintelligence service.
The Second World War
The Duquesne spy ring. William Sebold, a naturalized American, was coerced by the German Abwehr into spying while visiting Germany. He told the American consulate in Cologne and became a double agent for the FBI. In New York the Bureau set him up as a diesel engineer with an office in Manhattan, where agents filmed his meetings with German agents, and from May 1940 it ran a shortwave station on Long Island that passed Bureau-approved traffic to the Abwehr. In June 1941 the FBI arrested the network led by Frederick Joubert Duquesne. All 33 members pleaded guilty or were convicted at trial, and on 2 January 1942 they were sentenced to a combined total of more than 300 years (FBI). It remains the largest espionage case in American history to end in convictions.
The Pastorius saboteurs. In June 1942 German submarines landed eight trained saboteurs, four on Long Island and four in Florida. One of them, George Dasch, contacted the FBI and gave himself up, and all eight were in custody within two weeks. A military commission convicted them, the Supreme Court upheld the commission’s jurisdiction in Ex parte Quirin, 317 U.S. 1 (1942), and six were executed on 8 August 1942. Dasch and Ernest Burger, who had cooperated, had their death sentences commuted to prison terms (FBI).
The Special Intelligence Service. On 24 June 1940 Roosevelt gave the FBI responsibility for foreign intelligence in the Western Hemisphere, and Hoover formally established the Special Intelligence Service on 1 July. It was the first foreign-intelligence bureaucracy in American history, created before the Office of Strategic Services. Over seven years the Bureau sent more than 340 agents and support staff undercover into Central and South America; between 1941 and 1945 they located 24 clandestine German radio stations (FBI; Webb). The Central Intelligence Group replaced it in Latin America in 1946, and the Bureau’s postwar proposal for a worldwide FBI intelligence service was rejected. The agents’ liaison posts became the basis of the FBI’s Legal Attaché program.
The historian G. Gregg Webb, writing in the CIA’s journal, argues that the record does not support the common portrait of Hoover as an empire-builder in foreign intelligence. Hoover tried more than once to give the Special Intelligence Service up and accepted it reluctantly, Webb finds, though he also describes Hoover as often uncompromising and focused on protecting his institutional position (Webb).
The Cold War cases
Defectors and Venona. The Soviet code clerk Igor Gouzenko defected in Ottawa in September 1945, and Elizabeth Bentley, an American who had worked as a Soviet courier, came to the FBI that fall. Their accounts gave the Bureau its first detailed picture of wartime Soviet espionage (FBI). The decisive source was Venona, the Army’s project to read wartime Soviet intelligence cables, which ran from 1943 to 1980. In October 1948 FBI Special Agent Robert Lamphere began working full time with the Army codebreaker Meredith Gardner, and a large number of espionage cases were opened (NSA). The FBI credits Venona with helping identify some 350 people connected to Soviet intelligence. Because the decrypts were kept secret, they could point investigations in the right direction but could not be used in court; the Bureau had to build its evidence separately (NSA, Venona: An Overview).
The atom spies. British authorities arrested the physicist Klaus Fuchs on 2 February 1950. Fuchs identified his American courier, Harry Gold, who confessed to the FBI on 22 May 1950. Gold led to David Greenglass, and Greenglass to Julius and Ethel Rosenberg. A jury convicted the Rosenbergs and Morton Sobell on 29 March 1951; the Rosenbergs were executed on 19 June 1953 (FBI). In a separate case, a federal jury convicted the former State Department official Alger Hiss of perjury in January 1950 for denying that he had passed documents to Whittaker Chambers, and the conviction was affirmed on appeal (United States v. Hiss, 185 F.2d 822 (2d Cir. 1950)).
Judith Coplon. Not every case survived the courts. In March 1949 the FBI arrested Judith Coplon, a Justice Department analyst, as she met a Soviet employee of the United Nations. The trial judge in New York found that the Bureau had wiretapped her and that the wiretaps were unlawful even though the Attorney General had authorized them (United States v. Coplon, 88 F. Supp. 921 (S.D.N.Y. 1950)). The Second Circuit then reversed her New York conviction, holding that the arrest without a warrant was unlawful and that the government had not shown its wiretaps did not taint the evidence (United States v. Coplon, 185 F.2d 629 (2d Cir. 1950)). The D.C. Circuit upheld her Washington arrest but ordered a hearing on whether agents had listened to her calls with her lawyers, with a new trial required if they had (Coplon v. United States, 191 F.2d 749 (D.C. Cir. 1951)).
Rudolf Abel. In 1953 a Brooklyn newsboy was paid with a hollow nickel holding a microfilmed coded message. The FBI could not trace it for four years, until the Soviet illegal Reino Häyhänen defected in 1957 and led agents to his superior, the KGB officer known as Rudolf Abel. Abel was arrested on 21 June 1957, convicted, and sentenced to 30 years; in 1962 he was exchanged in Berlin for the U-2 pilot Francis Gary Powers (FBI).
Operation SOLO. Beginning in 1952 the FBI recruited the brothers Morris and Jack Childs, senior members of the Communist Party USA. For more than two decades, until 1977, the Childs brothers carried Soviet funds to the American party, met Soviet and Chinese leaders, and reported everything to the Bureau. The historians Harvey Klehr and John Earl Haynes, writing in the CIA’s journal, describe them as the most productive informants the FBI ever recruited (Klehr and Haynes, Studies in Intelligence, 2022).
The FBI and the CIA
The FBI’s Special Intelligence Service ended when the Central Intelligence Group, and then the CIA, took over foreign intelligence. Relations between the two agencies were often difficult. In 1966 they reached an informal agreement on CIA activity inside the United States. In March 1970, after a dispute over information sharing, Hoover ended direct liaison with the CIA at FBI headquarters and ordered that contact be by letter only, though working-level contacts continued (Church Committee, Book II). The agencies also judged sources differently: the FBI accepted as genuine the KGB defector Yuri Nosenko, whom the CIA’s counterintelligence staff under James Jesus Angleton believed was a plant.
Domestic intelligence and COINTELPRO
From 1956 to 1971 the FBI ran a series of covert action programs, known as COINTELPRO, aimed at groups inside the United States. The Church Committee identified five: the Communist Party USA program (1956–1971), the Socialist Workers Party program (1961–1969), the “White Hate Group” program aimed at the Ku Klux Klan and similar groups (1964–1971), the “Black Nationalist-Hate Group” program (1967–1971), and the “New Left” program (1968–1971). Bureau headquarters approved more than 2,000 separate actions (Church Committee, Book III). The programs ended in April 1971, after files stolen from the FBI’s office in Media, Pennsylvania, began appearing in the press.
The Committee found that the programs went beyond investigation to covert action against American citizens, that many targets were law-abiding, and that the programs interfered with rights protected by the First Amendment (Church Committee, Book II). It found that from late 1963 until his death in 1968, Martin Luther King Jr. was the target of an FBI campaign to discredit him, including wiretaps approved by the Attorney General, microphone surveillance, and an anonymous mailing in 1964 of a recording with a letter that King and his advisers read as urging him to suicide. In 1977 a federal court dismissed the damages claims brought by King’s former assistant and the Southern Christian Leadership Conference as time-barred, and ordered the surveillance tapes and transcripts sealed at the National Archives for 50 years (Lee v. Kelley, sealing order, 31 January 1977).
Hoover also restricted some techniques before any outside review. In 1965 he banned mail covers and trash covers. In 1966 he ended the FBI’s own mail-opening programs and its warrantless surreptitious entries other than microphone installations, and cut the number of warrantless wiretaps in half. In 1970 he opposed the Huston Plan, a White House proposal to lift those restrictions and create a permanent interagency domestic intelligence committee. He told Attorney General John Mitchell he would carry out such techniques only with specific approval from the Attorney General or the President in each case, and Mitchell advised the President to withdraw the plan (Church Committee, Book II).
After Hoover
Hoover died on 2 May 1972. Six weeks later, in United States v. United States District Court, 407 U.S. 297 (1972), the Supreme Court held that warrantless electronic surveillance in domestic security cases violated the Fourth Amendment. In 1980 a federal jury convicted two senior FBI officials, W. Mark Felt and Edward S. Miller, of conspiring to violate civil rights by authorizing warrantless break-ins in 1972 and 1973 during the search for Weather Underground fugitives; President Reagan pardoned both in April 1981 (Reagan Library).
Congress and the Justice Department rebuilt the rules around the Bureau. A 1968 law had already made the director a presidential appointee subject to Senate confirmation after Hoover. In 1976 Congress limited future directors to a single ten-year term, a change the FBI itself describes as a reaction to Hoover’s 48-year tenure (FBI, archived). Attorney General Edward Levi issued guidelines for FBI foreign counterintelligence and domestic security investigations in 1976, and the Foreign Intelligence Surveillance Act of 1978 put national-security surveillance inside the United States under a court.
What Hoover’s career teaches about counterintelligence
- Counterintelligence rests on institutions. The spy cases of the 1940s and 1950s depended on things built for other purposes: a national fingerprint file, a laboratory that could examine a hollow coin, trained agents in every field office, and records that could be searched. Capability built in peacetime decides what a service can do in a crisis.
- The best penetration is the adversary’s own channel. Sebold in the 1940s and the Childs brothers from the 1950s to the 1970s show what a controlled double agent can deliver: both intelligence on the adversary’s intentions and, when the time comes, evidence for a prosecution. The same idea ran Britain’s Double Cross system.
- Intelligence and evidence are different products. Venona identified spies, but its secrecy meant the FBI had to prove each case another way. Coplon was caught in the act and still won on appeal because of how the evidence was gathered. In a system governed by law, how you collect decides what you can use.
- Undefined targets grow. The authority Hoover received in 1936 to investigate “subversive activities” was never defined. The Church Committee traced how a mandate aimed at foreign intelligence services widened into covert action against American citizens. Angleton’s mail program at the CIA followed the same path. A counterintelligence service needs a definition of the threat that limits it as well as one that empowers it.
- Oversight is part of the design. Congress responded to Hoover’s tenure with term limits, confirmation, investigative guidelines, and a surveillance court. Those rules are now part of how American counterintelligence works, and any organization building a security function faces the same question in smaller form: who checks the people whose job is to watch?
For the psychology of how capable people misread an adversary, see Cognitive Bias: A Field Guide for Leaders. For what the same problems look like inside a company, see Corporate Counterintelligence.
Frequently asked questions
Who was J. Edgar Hoover?
J. Edgar Hoover (1895–1972) was the director of the Bureau of Investigation and then the Federal Bureau of Investigation from 1924 until his death in 1972, a tenure of 48 years. He built the FBI’s fingerprint, laboratory, training, and crime-statistics programs and led its counterintelligence work through the Second World War and the early Cold War.
What did Hoover contribute to American counterintelligence?
Under a 1939 presidential directive the FBI became the country’s domestic counterintelligence service. Under Hoover it broke the Duquesne spy ring, captured the Pastorius saboteurs, ran the Special Intelligence Service in Latin America, worked the Venona decrypts with the Army codebreakers, investigated the atomic espionage cases, caught Rudolf Abel, and ran Operation SOLO inside the Communist Party USA.
What was COINTELPRO?
COINTELPRO was a set of FBI covert action programs run from 1956 to 1971 against domestic groups, including the Communist Party USA, the Socialist Workers Party, the Ku Klux Klan, Black nationalist groups, and the New Left. The Senate’s Church Committee documented it in 1976 and found that it targeted many law-abiding Americans and interfered with First Amendment rights.
Why is the FBI director’s term now limited to ten years?
In 1976, after Hoover’s 48 years in office, Congress limited future FBI directors to a single ten-year term. A 1968 law had already required that directors appointed after Hoover be confirmed by the Senate.
How did Hoover’s FBI get along with the CIA?
The FBI ran American foreign intelligence in Latin America during the Second World War through its Special Intelligence Service, which the Central Intelligence Group replaced in 1946. Relations with the CIA were often strained, and in 1970 Hoover ended direct liaison with the CIA at FBI headquarters. The two agencies also disagreed over the Soviet defector Yuri Nosenko, whom the FBI accepted as genuine.
Work with Competitive Futures
We work on the problem at the center of Hoover’s counterintelligence record: finding out what a capable adversary is doing, within the law, and with evidence that holds up. Request a 20-minute conversation.
Related analysis from Competitive Futures
- James Jesus Angleton: Counterintelligence in a Wilderness of Mirrors
- Richards Heuer: The CIA Officer Who Taught Analysts How Their Minds Fail
- Corporate Counterintelligence
- The Economic Espionage Act
- The Foreign Agents Registration Act
References
- Colyer v. Skeffington, 265 F. 17 (D. Mass. 1920).
- Coplon v. United States, 191 F.2d 749 (D.C. Cir. 1951).
- Federal Bureau of Investigation. J. Edgar Hoover (director biography); FBI timeline; World War, Cold War, 1939–1953; famous cases: Duquesne Spy Ring, Nazi Saboteurs and George Dasch, Atom Spy Case/Rosenbergs, Hollow Nickel/Rudolf Abel. https://www.fbi.gov/history
- Fox, J. F., Jr. The birth of the FBI’s Technical Laboratory, 1924 to 1935. FBI.
- Klehr, H., & Haynes, J. E. (2022). Running SOLO: FBI’s case of Morris and Jack Childs, 1952–77. Studies in Intelligence, 66(1).
- Lee v. Kelley and SCLC v. Kelley, Civ. Nos. 76-1185 and 76-1186 (D.D.C., memorandum opinion and order, 31 January 1977).
- National Security Agency, Center for Cryptologic History. Venona chronology; Venona: An Overview.
- Pub. L. No. 90-351 (1968), Title VI; Pub. L. No. 94-503, § 203 (1976).
- United States v. Coplon, 88 F. Supp. 921 (S.D.N.Y. 1950); 185 F.2d 629 (2d Cir. 1950).
- United States v. Hiss, 185 F.2d 822 (2d Cir. 1950).
- United States v. United States District Court, 407 U.S. 297 (1972).
- Ex parte Quirin, 317 U.S. 1 (1942).
- U.S. General Accounting Office. (1976). FBI Domestic Intelligence Operations: Their Purpose and Scope (GGD-76-50).
- U.S. Senate, Select Committee to Study Governmental Operations with Respect to Intelligence Activities. (1976). Final Report, Book II: Intelligence Activities and the Rights of Americans; Book III: Supplementary Detailed Staff Reports.
- Webb, G. G. The FBI and foreign intelligence: New insights into J. Edgar Hoover’s role. Studies in Intelligence, 48(1).
This piece is part of our guide to The Pioneers of Strategy, Intelligence, and Foresight.