Why were the Allies so much better cryptanalysts?

Why were the Allies so much better cryptanalysts?

We are searching data for your request:

Forums and discussions:
Manuals and reference books:
Data from registers:
Wait the end of the search in all databases.
Upon completion, a link will appear to access the found materials.

When I read about World War II, more specifically about the use of cryptography in that war, I get the impression that the Allies were much more successful in breaking the enemy codes than the Axis Powers. Two such successes are particulary important: the breaking of the Enigma code by the Polish and the British and the breaking of the Purple code by the British and the Americans.

My question is: why were the Allies so much better here? Of course, a possibility is that they had a larger pool of highly skilled people. But is that all? Or is it also because their governments decided to bet more on that than the governments of the Axis Powers?

Edit: Thanks to the comments, I am now aware of some successes of the German code breaking and also of the fact that the Germans never had a central cryptography agency. But why is that? And what about Japan?

Short Answer

Allied superiority in cryptography versus both the Germans and the Japanese can be broadly attributed to (1) better/greater coordination among personnel, awareness of vulnerabilities, and allocation of resources for breaking enemy codes and, (2) the fact that Axis codes were (mostly, though not always) more easy to break than Allied ones. However, it should not be overlooked that the Axis powers did break some important Allied codes, and that some Axis codes were not broken by the Allies.

German Codebreaking vs. Allied Codebreaking

While the Allies were superior in the use of cryptography, the Germans had their successes as well, as detailed in the Wikipedia article German code breaking in World War II. The most notable success was the breaking of many British naval codes by B-Dienst, which proved particularly damaging to transatlantic convoys.

Nonetheless, there was overall Allied superiority in this area and this has been attributed to nine factors by David Kahn in his article Codebreaking in World Wars I and II: The Major Successes and Failures, Their Causes and Their Effects (1980). Kahn divides these nine reasons into two broad categories: (1) internal or technical and, (2) external or general.

For internal or technical, and listed in rough order of importance, Kahn notes:

Allied knowledge of the Enigma: the German use of one main machine versus the Allied use of many; a poorer German machine; and inadequate operating procedures.

On the first of these points,

The Enigma was originally sold to the public. Even though it was modified for government use, and even though the several agencies of government had their own variations of it, the Allies knew its basic layout. To this must be added the information about its keys and operation provided by the spy. Cryptanalytically, this is of course an enormous head start. It is also a great psychological advantage. The Germans did not have these benefits. The British Type-X and the American SIGABA machines were developed in secret.

On the second point,

This use of one machine [Enigma, by the Germans] had several effects. First of all, it meant that the Allies could concentrate more manpower on a single problem. Secondly, the greater volume of messages enciphered in that single system facilitated its solution. Thirdly, a single system increased Allied incentive, because its solution would yield a greater prize than if it were just one system among many. None of these factors operated for the Germans, and it correspondingly depressed their efforts and results.

On the third point, and referring to the American SIGABA,

It was in fact devised a decade after the Enigma, and because the Americans did not begin equipping their army and navy with cipher machines until the late 1930s, they could utilize this more advanced mechanism without losing capital investment. The Germans, who had mechanized a decade earlier, were stuck with an older, weaker machine.

On the fourth point,

… just as the German hardware was poorer, so was their software. Two of their operating procedures proved fatal to many an Enigma cryptogram. One was the flawed keying method used by the Germans before the war and for its first year or so… This keying method was later changed, but by then Enigma had been cracked. The Allies, on the other hand, used far more secure keying systems which obviated this sort of attack.

Turning to the five external or general reasons,

… the fragmentation of the German organization compared to the unity of the Allied; Germany's aggression, which led to a neglect of cryptology, contrasted with the Allied defensive posture, which emphasised intelligence; the expulsion and killing of the Jews; better Allied luck, and greater German reluctance to face reality.

Kahn asserts that the first of these was the most important:

The Germans had a great many codebreaking agencies. The Chiffrierabteilung of the armed forces high command, Pers Z of the Foreign Office, and Goring's Forschungsamt competed on the highest level. For a time the SD, the Sicherheitsdienstt, the SS's intelligence arm, had its own agency. The army, the navy, and the air force each had its own unit, though there was rather more justification for that. But this multiplicity spread the available manpower, which was scarce to begin with, very thin. And it diffused the codebreaking effort. Contrast this with the concentration of effort at Bletchley Park, Britain's sole codebreaking agency, and with that in America, where the army and navy codebreaking units worked in the closest co-operation.

On the other points, Kahn notes that German recruiting and training were inferior to that of the Allies while, concerning the Jews, the

… exodus or extermination of a whole people, many of them highly intelligent, cost German codebreaking - as it cost German mathematics and German physics - many useful brains.

Finally, the Germans were slow to accept that Enigma had been broken, even when faced with strong evidence. A senior British MI6 officer, F. W. Winterbotham, noted this when he later wrote in The Ultra Secret (1974) that the Germans

… must have been puzzled by our knowledge of their U-boat positions, but luckily they did not accept the fact that we had broken enigma.

Japanese Codebreaking vs. Allied Codebreaking

A key point that needs emphasising here, even more so than for the Germans, is not that the Japanese were inept at codebreaking but rather that they made it much easier for the Americans to break their codes than it should have been. This was especially true of the Imperial Japanese Navy:

William Friedman, the great American cryptologist… noted that while high level US Naval communications security in WW2 was quite adequate for the time, Japanese Naval communications security was quite inadequate and the IJN lacked the 'experience and knowledge' to rectify it.

Source: Peter Donovan & John Mack, 'Code Breaking in the Pacific' (2014)

The Japanese did, though, have a significant amount of success in cracking Allied codes:

… the SIGINT abilities of Japan were not as low as has traditionally been thought. The Imperial Japanese Army was able to read the diplomatic codes of the US, Great Britain, France and China, and some of the military codes of China and the Soviet Union.

Source: Kotani Ken, 'Japanese Intelligence in WWII: Successes and Failures' (NIDS Security Reports, 2009)

Also, unlike Japanese Imperial Navy codes,

In the case of the Army, counterintelligence activities were relatively effective, and there were no cases of Army codes being deciphered by the Allies until the final stages of the war.

Source: Kotani Ken

A key reason for the weakness of the Imperial Navy on the intelligence front was that they effectively ignored signs that their codes had been broken but a

… thorough investigation of the cause and countermeasures were not implemented… Rear Admiral Ryunosuke Kusaka, who had participated with the assistant chief of staff of the 1st air fleet, stated that “The fact that the planning of the Combined Fleet in relation to the Battle of Midway was leaked to the US side was a major cause of the failure of that operation.” In an Imperial Japanese Navy General Staff diary, it was stated that “the enemy had sensed our plan.” However, even though these suspicions remained in the Imperial Japanese Navy General Staff, the cause of defeat at Midway was basically considered to be technical operational factors, such as problems in cooperation with supply ships and inadequacy in searching for the enemy. Ultimately, the fact that the Japanese codes had been deciphered was not touched upon.

Source: Kotani Ken

One problem for the Japanese was the far-flung nature of their empire; this made it difficult to easily implement code changes. Further,

Commander Chikataka Nakajima who was a specialist in communications within the Navy recalls that “the greatest deficiency in our Navy's coding plan was inadequate consideration of the fact that our code charts could fall into enemy hands.”

The weak awareness of counterespionage on the part of the Imperial Japanese Navy at the time, and the lack of a self-cleansing function caused a number of problems to arise. When we consider the effects exerted on subsequent naval strategies, they were all serious. Even if one of the codes were taken, the arrogance that “our codes cannot be deciphered” meant that little labor was put into counterintelligence work.

Source: Kotani Ken

Some of these weaknesses would probably have been dealt with had the Japanese army and navy co-operated and learnt from each other (especially the latter from the former). However, as gktscrk pointed out in a comment below, interservice rivalry was fierce, the damage stretching far beyond counter-intelligence.

As for the Japanese failure to break Navajo (and, as noted by jamesqf in a comment below, other Native American) codes, these were exceptionally difficult to crack:

The Navajo language seemed to be the perfect option as a code because it is not written1 and very few people who aren't of Navajo origin can speak it.

However, the Marine Corps took the code to the next level and made it virtually unbreakable by further encoding the language with word substitution.

Source: 'Navajo Code Talkers and the Unbreakable Code' (CIA)

A further complication for the Japanese was that it was used in the field via portable radios so, without a Navajo speaker to hand, this made it even more secure:

The Navajo language has no definite rules and a tone that is guttural. The language was unwritten at the time1, notes Carl Gorman, one of the 29 original Navajo code talkers. "You had to base it solely on the sounds you were hearing," he says. "This made it very difficult for others to understand."

1. The 'not written' and 'unwritten' bits are not strictly true (see, for example, Schwern's answer here and this article, but it wasn't widely available and new words had to be invented for military terms.

The Deceivers by Thaddeus Holt is a history of Allied deception operations in WWII, but it necessarily also covers codebreaking. I'm not sure how objective the book is (it seems to be rehashing some British/American feuds) but it was quite dismissive of Japanese intelligence operations in general. Lots of Allied (British) effort to plant false clues and then the Japanese didn't notice, and instead got the right answers by asking themselves "what is the logical next step for the Allies?"

The Jewish expulsion from Germany is mentioned which might have had an effect but I would suggest that another related effect might be involved: people were put into positions of authority in academia and scientific research based not so much on ability but on their loyalty to the nazi party. Although some capable mathematicians and scientists were indeed loyal nazis, perhaps nazis who were even more loyal ended up being influential despite being not nearly as competent and this could have affected as it did math and physics also the cryptology effort.

Two nazis who were also Nobelists so very competent physicists were Lenard and Stark but both were from the older generation and did their best work in the 19th century. They ended up being influential in "German Physics" but were far from being up-to-date in their understanding of modern physics.

The question then is, who was in charge of the cryptology effort? Was it a nazi mathematician who was not the most competent but only the most loyal? Or could it even have been a military officer familiar with codes used in the ww1 and lacking the mathematical knowledge to even realize that the effort required first-rate mathematicians? Doenitz apparently compromised the codes by using too often and unnecessarily but in a dictatorship, questioning this practice might have been dangerous.