2
$\begingroup$

I am experimenting with Keras and I managed to build a simple CNN to classify blue images (300x300 images of the same shade of blue) vs. red images (same size, just red). This is a dummy problem I assumed the NN would have solved immediately but this doesn't seem the case. In fact, even after 20+ epochs accuracy is still exactly 50%.

I assume there's a lot of stuff I could try to do differently but is there anything extraordinary wrong I am doing here that could result in such poor performance on such an easy task?

# Create a Keras model.
model = keras.Sequential()
model.add(
    keras.layers.Conv2D(
        input_shape=(300, 300, 3),
        filters=64,
        kernel_size=(3, 3),
        activation='relu',
    )
)
model.add(
    keras.layers.Conv2D(
        filters=64,
        kernel_size=(3, 3),
        activation='relu',
    )
)
model.add(
    keras.layers.MaxPooling2D(
        pool_size=(2, 2),
        strides=(2, 2),
    )
)
model.add(keras.layers.Flatten())
model.add(
    keras.layers.Dense(
        units=128,
        activation='relu'
    )
)
model.add(keras.layers.Dropout(0.5))
model.add(
    keras.layers.Dense(
        units=1,
        activation='sigmoid',
    )
)

# Train the model.
sgd = keras.optimizers.SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(
    optimizer=sgd,
    loss='mean_squared_error',
    metrics=['binary_accuracy']
)
model.fit(images, labels, epochs=20, batch_size=5)

Which outputs:

20/20 [==============================] - 18s 887ms/step - loss: 0.5954 - binary_accuracy: 0.4000
Epoch 2/20
20/20 [==============================] - 16s 794ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 3/20
20/20 [==============================] - 16s 781ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 4/20
20/20 [==============================] - 17s 853ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 5/20
20/20 [==============================] - 18s 877ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 6/20
20/20 [==============================] - 18s 891ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 7/20
20/20 [==============================] - 17s 825ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 8/20
20/20 [==============================] - 17s 861ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 9/20
20/20 [==============================] - 17s 846ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 10/20
20/20 [==============================] - 17s 835ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 11/20
20/20 [==============================] - 16s 800ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 12/20
20/20 [==============================] - 16s 806ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 13/20
20/20 [==============================] - 16s 811ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 14/20
20/20 [==============================] - 17s 827ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 15/20
20/20 [==============================] - 16s 806ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 16/20
20/20 [==============================] - 16s 786ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 17/20
20/20 [==============================] - 16s 795ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 18/20
20/20 [==============================] - 16s 796ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 19/20
20/20 [==============================] - 16s 788ms/step - loss: 0.5000 - binary_accuracy: 0.5000
Epoch 20/20
20/20 [==============================] - 16s 794ms/step - loss: 0.5000 - binary_accuracy: 0.5000
$\endgroup$
  • $\begingroup$ Omit the dropout and add another dense layer with multiple neurons. $\endgroup$ – Media Jul 20 '18 at 19:36
  • $\begingroup$ You have too many filters. Reduce it to 32 or 16. Also, make sure your data is all standardized. $\endgroup$ – Jon Jul 22 '18 at 2:56
  • $\begingroup$ @Jon Any recommended way to standardize? $\endgroup$ – Edgar Derby Jul 22 '18 at 6:44
  • $\begingroup$ Kinda depends on your data. But generally, $(X - \mu)/\sigma)$ works well. $\endgroup$ – Jon Jul 22 '18 at 6:46
  • $\begingroup$ @EdgardDerby subtract by your mean and divide by the standard deviation $\endgroup$ – Jon Jul 22 '18 at 6:47
2
$\begingroup$

If you remember the theory, CNN kernels basically work on the principle of edge detection. So now that your picture is purely blue or purely red there are no edges. I am guessing that is the main problem here, since I think ML libraries initialise kernels which are suitable for edge detection. To be more comprehensive:

  • Since kernels are randomised small numbers, sum of red kernel is almost equal to sum of blue kernel. The next layer will mainly have (sum of red kernel * 255) and (sum of blue kernel * 255) in all the channels for the next layer, which will be roughly equal. Since you are using an inappropriate loss function, the CNN becomes insensitive to small changes due to the difference in summation of kernels.

I would suggest:

  • Changing the padding to same might work since then edges will be introduced in the picture (black-red, black - blue).
  • Try by adding significant white edges to the side of the picture.
  • Squared error does not work that good for sigmoid activations, you need to use the cross entropy loss.
  • One must i believe experimenting with smaller learning rates, since CNN have the problem of exploding gradients.
$\endgroup$
  • $\begingroup$ This is brilliant, thanks a lot for the reply. I'll run some tests and come back with more insights! $\endgroup$ – Edgar Derby Jul 20 '18 at 20:44
  • $\begingroup$ Okay, so: I switched to pictures of the number 3 and 7, experimented with and without padding, changed the loss to binary_crossentropy, reduce the LR but... Still nothing, accuracy doesn't move from 50%. $\endgroup$ – Edgar Derby Jul 20 '18 at 20:59
  • $\begingroup$ @EdgardDerby i have not used keras but i would assume it is similar to tensorflow so i assume no logical error, i had not previously noticed but u used sigmoid in the final layer (1 unit) sigmoid is quite a dodgy customer, either use 2 sigmoid units in the final layer or 2 softmax units, also check the input to the final layer by printing out the input...are they equal?? what did you mean by switched pics of number 3 and 7? $\endgroup$ – DuttaA Jul 21 '18 at 1:26
  • $\begingroup$ @EdgardDerby i asked the same question stackoverflow.com/questions/51278084/… it somewhat worked for me By using a learning rate 0.00001 and I used an adam optimizer. $\endgroup$ – DuttaA Jul 21 '18 at 1:30
1
$\begingroup$

A few things:

  1. Are you sure your data / labels are set up correctly?
  2. Is every example of each class exactly the same?
  3. Most importantly, why are you using mean squared error? Shouldn't your target values be 0 and 1, and your loss function be binary cross entropy?
$\endgroup$
  • $\begingroup$ Thanks for the reply! So, in order: (1) Yes. (2) Yes. (3) My bad, I switched to binary_crossentropy. $\endgroup$ – Edgar Derby Jul 20 '18 at 20:55
0
$\begingroup$

Based on your comments and your overall architecture, take a look at your classification layers, dense layers. You just have one neuron which is just a linear separator of the extracted features. Moreover, you are dropping the extracted features and feed them to that linear separator with rate $0.5$ which means you are destroying the signal. Try not to use dropout first in order to find a model that learns your data well or even memorise it to find a model that is appropriate for your task and also add at least one more dense layer after the flatten layer and before the last layer with maybe $10$ neurons to classify non-linear data points. It seems that your task is classification, so keep using binary-cross-entropy.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.