Data visualization for pattern analysis (language-independent, but R preferred)

Question

I want to plot the bytes from a disk image in order to understand a pattern in them. This is mainly an academic task, since I'm almost sure this pattern was created by a disk testing program, but I'd like to reverse-engineer it anyway.

I already know that the pattern is aligned, with a periodicity of 256 characters.

I can envision two ways of visualizing this information: either a 16x16 plane viewed through time (3 dimensions), where each pixel's color is the ASCII code for the character, or a 256 pixel line for each period (2 dimensions).

This is a snapshot of the pattern (you can see more than one), seen through xxd (32x16):

Either way, I am trying to find a way of visualizing this information. This probably isn't hard for anyone into signal analysis, but I can't seem to find a way using open-source software.

I'd like to avoid Matlab or Mathematica and I'd prefer an answer in R, since I have been learning it recently, but nonetheless, any language is welcome.

---

Update, 2014-07-25: given Emre's answer below, this is what the pattern looks like, given the first 30MB of the pattern, aligned at 512 instead of 256 (this alignment looks better):

Any further ideas are welcome!

Answer 1

I would use a visual analysis. Since you know there is a repetition every 256 bytes, create an image 256 pixels wide by however many deep, and encode the data using brightness. In (i)python it would look like this:

import os, numpy, matplotlib.pyplot as plt

%matplotlib inline

def read_in_chunks(infile, chunk_size=256):
    while True:
        chunk = infile.read(chunk_size)
        if chunk:
            yield chunk
        else:
            # The chunk was empty, which means we're at the end
            # of the file
            return

fname = 'enter something here'
srcfile = open(fname, 'rb')
height = 1 + os.path.getsize(fname)/256
data = numpy.zeros((height, 256), dtype=numpy.uint8)    

for i, line in enumerate(read_in_chunks(srcfile)):
    vals = list(map(int, line))
    data[i,:len(vals)] = vals

plt.imshow(data, aspect=1e-2);

This is what a PDF looks like:

A 256 byte periodic pattern would have manifested itself as vertical lines. Except for the header and tail it looks pretty noisy.

Answer 2

I know almost nothing about signal analysis, but 2-dimensional visualization could be easily done using R. Particularly you will need reshape2 and ggplot2 packages. Assuming your data is wide (e.g. [n X 256] size), first you need to transform it to long format using melt() function from reshape2 package. Then use geom_tile geometry from ggplot2. Here is a nice recipe with gist.

Answer 3

I would look at the raster package for this, which can read in raw binary data and present it as NxM grids. It can even extract subsets of large binary grids without having to read in the whole file (the R raster object itself is just a proxy to the data, not the data itself).