I was reading about Hamming Distance. From what I understand, the key length can be guessed fairly accurately by comparing each hex pair. If the hex pairs are of similar Hamming Distances, then they are most likely encoded with the same hex pair. So, if they were 10 characters apart, then the key length is 10. I need to do further reading about this and experiment with it to see if I can better understand it.
After searching Google for a while, I stumbled upon a tool called "XorTool" on GitHub. I'm using VM's, so I downloaded, scanned, and installed it. Then I set about learning how to use it. I was given the hint that the key length may be 10. I was also told that the plaintext was a "history of cryptography", so I had a good idea of what I was looking for. I let the tool do the work for me. I just typed, xortool -x -l 10 encrypted. -x told the program that the file was hex encoded, -l told the program that I was guessing a key length of 10, and encrypted was the name of the encrypted file. It guessed that the most likely length was 10. So then I ran the following command and got the following output.
$ xortool -x -o encrypted
The most probable key lengths:
2: 9.7%
5: 14.5%
8: 7.2%
10: 20.7%
12: 6.0%
15: 8.9%
20: 12.8%
25: 5.7%
30: 8.5%
40: 6.1%
Key-length can be 5*n
100 possible key(s) of length 10:
\x94\xd6\xb1\xc2\xbc\t\x05\xd6\x1c6
\x95\xd7\xb0\xc3\xbd\x08\x04\xd7\x1d7
\x96\xd4\xb3\xc0\xbe\x0b\x07\xd4\x1e4
\x97\xd5\xb2\xc1\xbf\n\x06\xd5\x1f5
\x90\xd2\xb5\xc6\xb8\r\x01\xd2\x182
...
Found 51 plaintexts with 95.0%+ printable characters
See files filename-key.csv, filename-char_used-perc_printable.csv
After this, I read filename-char_used-perc_printable.csv. This gave me a decent idea of which keys were correct, because it told me the percentage of the characters in each potential key that were printable. Xortool saves possible plain text files as out files. I navigated to the folder that contains these out files. I just used cat <numberIwasinterestedin>.out in my terminal, and it printed out the out file. I only printed the texts with 100 percent printable characters. There were only 7, so it made finding the correct decrypted file really easy.
$cat 94.out
your flag is: ab2614e35e828a602c50ebc9b0f5d710e2312388
On 17 March 1975, the proposed DES was published in the Federal Register. Public comments were requested, and in the following year two open workshops were held to discuss the proposed standard. There was some criticism from various parties, including from public-key cryptography pioneers Martin Hellman and Whitfield Diffie, citing a shortened key length and the mysterious "S-boxes" as evidence of improper interference from the NSA. The suspicion was that the algorithm had been covertly weakened by the intelligence agency so that they - but no-one else - could easily read encrypted messages. Alan Konheim (one of the designers of DES) commented, "We sent the S-boxes off to Washington. They came back and were all different." The United States Senate Select Committee on Intelligence reviewed the NSA's actions to determine whether there had been any improper involvement. In the unclassified summary of their findings, published in 1978, the Committee wrote:
In the development of DES, NSA convinced IBM that a reduced key size was sufficient; indirectly assisted in the development of the S-box structures; and certified that the final DES algorithm was, to the best of their knowledge, free from any statistical or mathematical weakness.
However, it also found that
NSA did not tamper with the design of the algorithm in any way. IBM invented and designed the algorithm, made all pertinent decisions regarding it, and concurred that the agreed upon key size was more than adequate for all commercial applications for which the DES was intended.
Another member of the DES team, Walter Tuchman, stated "We developed the DES algorithm entirely within IBM using IBMers. The NSA did not dictate a single wire!" In contrast, a declassified NSA book on cryptologic history states:
In 1973 NBS solicited private industry for a data encryption standard (DES). The first offerings were disappointing, so NSA began working on its own algorithm. Then Howard Rosenblum, deputy director for research and engineering, discovered that Walter Tuchman of IBM was working on a modification to Lucifer for general use. NSA gave Tuchman a clearance and brought him in to work jointly with the Agency on his Lucifer modification."
and
NSA worked closely with IBM to strengthen the algorithm against all except brute force attacks and to strengthen substitution tables, called S-boxes. Conversely, NSA tried to convince IBM to reduce the length of the key from 64 to 48 bits. Ultimately they compromised on a 56-bit key.
Some of the suspicions about hidden weaknesses in the S-boxes were allayed in 1990, with the independent discovery and open publication by Eli Biham and Adi Shamir of differential cryptanalysis, a general method for breaking block ciphers. The S-boxes of DES were much more resistant to the attack than if they had been chosen at random, strongly suggesting that IBM knew about the technique in the 1970s. This was indeed the case; in 1994, Don Coppersmith published some of the original design criteria for the S-boxes. According to Steven Levy, IBM Watson researchers discovered differential cryptanalytic attacks in 1974 and were asked by the NSA to keep the technique secret. Coppersmith explains IBM's secrecy decision by saying, "that was because [differential cryptanalysis] can be a very powerful tool, used against many schemes, and there was concern that such information in the public domain could adversely affect national security." Levy quotes Walter Tuchman: "[t]hey asked us to stamp all our documents confidential... We actually put a number on each one and locked them up in safes, because they were considered U.S. government classified. They said do it. So I did it". Bruce Schneier observed that "It took the academic community two decades to figure out that the NSA 'tweaks' actually improved the security of DES."
