Theano/Lasagne/Nolearn Neural Network Image Input

Question

I am working on image classification tasks and decided to use Lasagne + Nolearn for neural networks prototype. All standard examples like MNIST numbers classification run well, but problems appear when I try to work with my own images.

I want to use 3-channel images, not grayscale. And there is the code where I'm trying to get arrays from images:

 img = Image.open(item)
 img = ImageOps.fit(img, (256, 256), Image.ANTIALIAS)
 img = np.asarray(img, dtype = 'float64') / 255.
 img = img.transpose(2,0,1).reshape(3, 256, 256)   
 X.append(img)

Here is the code of NN and its fitting:

X, y = simple_load("new")

X = np.array(X)
y = np.array(y)


net1 = NeuralNet(
    layers=[  # three layers: one hidden layer
        ('input', layers.InputLayer),
        ('hidden', layers.DenseLayer),
        ('output', layers.DenseLayer),
        ],
    # layer parameters:
    input_shape=(None, 65536),  # 96x96 input pixels per batch
    hidden_num_units=100,  # number of units in hidden layer
    output_nonlinearity=None,  # output layer uses identity function
    output_num_units=len(y),  # 30 target values

    # optimization method:
    update=nesterov_momentum,
    update_learning_rate=0.01,
    update_momentum=0.9,

    regression=True,  # flag to indicate we're dealing with regression problem


       max_epochs=400,  # we want to train this many epochs
        verbose=1,
        )

  net1.fit(X, y)

I recieve exceptions like this one:

Traceback (most recent call last):
  File "las_mnist.py", line 39, in <module>
    net1.fit(X[i], y[i])
  File "/usr/local/lib/python2.7/dist-packages/nolearn/lasagne.py", line 266, in fit
    self.train_loop(X, y)
  File "/usr/local/lib/python2.7/dist-packages/nolearn/lasagne.py", line 273, in train_loop
    X, y, self.eval_size)
  File "/usr/local/lib/python2.7/dist-packages/nolearn/lasagne.py", line 377, in train_test_split
    kf = KFold(y.shape[0], round(1. / eval_size))
IndexError: tuple index out of range

So, in which format do you "feed" your networks with image data? Thanks for answers or any tips!

Answer 1

Just out of curiousity: why use 3-channel images? I work in CV as well and from what I've seen, it's standard to grayscale the images. Especially with MNIST, which is not colored to begin with, it doesn't seem that using color has any benefit.

I also believe that there's an intuitive reason to use grayscale - often times color is a confounding variable. If you were to look at a red "1" and and blue "1", you'd say "Hey! Those are both ones! They're just different colors". However, computers are much stupider than you or I. Even with intelligent algorithms, a computer may still end up in the situation where it's looking at a blue "1" and say to itself, "Oh crap, what is that?!? Why is it blue? I've never seen anything like this before!" Keep in mind how large of a difference there is from the color (255, 0, 0) and (0, 255, 255) and how slight the differences in images can be (especially outside the comfortable playground of MNIST).

At any rate, most of the NoLearn examples I've seen use data shapes like:

X = X.reshape(-1, 1, size, size)

If you grayscale your images you can coerce it into this shape. Unfortunately, I'm not sure of how you'd cram your colored data into noLearn and get the results you're looking for.