Tuesday, April 1, 2008

Binary File Visual Analysis Redux

I got a great comment on my post regarding simple binary file visual analysis from Erik Heidt. Erik made the very valid point that visual analysis of ciphertext is not a highly reliable way to distinguish "good" crypto from "bad." He used the example of an 8-bit XOR of a file as an ineffective method of encrypting data that also has random byte distribution.

Since there's nothing good on TV, I decided to see what an XOR-ed file data looks like in gnuplot. So here's what I did.

Like before, I used the Netcat nc.exe binary. I then encrypted it using GPG and also encoded it using Luigi Auriemma's Xor utility. I then ran the three files through the Perl script from my previous post and then plotted them with gnuplot.

Here's the plot of the original binary:

Here's the plot of the GPG-encrypted file:

And here's the plot of the XOR-encoded file:

As you can see, the XOR plot has peaks and valleys that are characteristically similar to the original binary. I don't want you to take away from this that this visual analysis method is highly reliable in all situations. I only wanted to share that basic XOR encoding does stand out visually.


Wesley McGrew said...

Very cool work :)

Calculating the data's index-of-coincidence for various shifts is also a pretty nice quick-and-dirty way of seeing how "random" it is. One of the first posts to my blog covers playing around with IoC in python:


nate said...

The reason an XOR leaves that trace is you can really think of it as a pure permutation. If you took the bytes in the file and shuffled them around, they'd have the same frequency distribution if you XORd them all with some random 8-bit value.

You can think of an 8-bit XOR as a bytewise lookup table. Same as saying "take all the 0x7d and replace with 0x28, all the 0x7e and replace with 0xea", etc.

Erik Heidt said...

Paul -

Is is possible for you to add CBC to the XOR function that you used? I think my point was that XOR with CBC would look very random.

I will try to find some quick code to do this... If I find it I will post the code... (As I don't see a way to add the CBC to the Luigi Auriemma utility.)

Cheers, Erik