I'm trying to train a CNN to play an online game by feeding images of the game along with the keyboard input.

By playing the game for some time and collecting the data, I gathered 342 images with size 110x42. I'm feeding these images in the network like so:

def convolutional_neural_network(x):
    weights = {'W_conv1': tf.Variable(tf.random_normal([3, 3, 1, 16])),
               'W_conv2': tf.Variable(tf.random_normal([5, 5, 16, 32])),
               'W_conv3': tf.Variable(tf.random_normal([5, 5, 32, 64])),
               'W_conv4': tf.Variable(tf.random_normal([5, 5, 64, 64])),
               'W_fc': tf.Variable(tf.random_normal([7 * 3 * 64, 1024])),
               'out': tf.Variable(tf.random_normal([1024, n_classes]))}

    biases = {'b_conv1': tf.Variable(tf.random_normal([16])),
              'b_conv2': tf.Variable(tf.random_normal([32])),
              'b_conv3': tf.Variable(tf.random_normal([64])),
              'b_conv4': tf.Variable(tf.random_normal([64])),
              'b_fc': tf.Variable(tf.random_normal([1024])),
              'out': tf.Variable(tf.random_normal([n_classes]))}

    x = tf.reshape(x, shape=[-1, 110, 42, 1])

    conv1 = tf.nn.relu(conv2d(x, weights['W_conv1']) + biases['b_conv1'])
    conv1 = maxpool2d(conv1)

    conv2 = tf.nn.relu(conv2d(conv1, weights['W_conv2']) + biases['b_conv2'])
    conv2 = maxpool2d(conv2)

    conv3 = tf.nn.relu(conv2d(conv2, weights['W_conv3']) + biases['b_conv3'])
    conv3 = maxpool2d(conv3)

    conv4 = tf.nn.relu(conv2d(conv3, weights['W_conv4']) + biases['b_conv4'])
    conv4 = maxpool2d(conv4)

    fc = tf.reshape(conv3, [-1, 7 * 3 * 64])
    fc = tf.nn.relu(tf.matmul(fc, weights['W_fc']) + biases['b_fc'])
    fc = tf.nn.dropout(fc, keep_rate)

    output = tf.sigmoid(tf.add(tf.matmul(fc, weights['out']), biases['out'], name='pred'))

    return output

def train_neural_network(x):
    prediction = convolutional_neural_network(x)
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=prediction, labels=y))
    optimizer = tf.train.AdamOptimizer(learning_rate=1).minimize(cost)

    hm_epochs = 6
    with tf.Session() as sess:

        for epoch in range(hm_epochs):
            epoch_loss = 0
            for epoch_x, epoch_y, i in dataset.create_batches():
                epoch_x = epoch_x.reshape(-1,4620)
                _, c = sess.run([optimizer, cost], feed_dict={x: epoch_x, y: epoch_y})
                epoch_loss += c

            print('Epoch', epoch, 'completed out of', hm_epochs, 'loss:', epoch_loss)

        #correct = tf.equal(tf.argmax(prediction, 1), tf.argmax(y, 1))
        correct = tf.equal(tf.greater(prediction, 0.5), tf.equal(y, 1.0))
        accuracy = tf.reduce_mean(tf.cast(correct, 'float'))
        print('Accuracy:', accuracy.eval({x: dataset.dataset['test']['x_test'], y: dataset.dataset['test']['y_test']}))


The errors are getting stuck in a particular value and floating up and down around this value by a small amount.

I've tried incresing/decreasing the learning rate, improving the quality of images, changing the size of the batches... and nothing seems to make the network stable.

Do you guys know what I'm doing wrong?

  • 1
    $\begingroup$ What learning rates did u use? BTW your net is very shallow and also try to use sigmoid before the final classifying layer $\endgroup$
    – DuttaA
    Jul 31, 2018 at 0:43
  • $\begingroup$ I tried from very small (like 0.002) to very big like 100. And all returned the same type of result. $\endgroup$
    – WilsonPena
    Jul 31, 2018 at 0:45
  • 1
    $\begingroup$ 0.002 is not small in case of CNN 0.00001 is the general case, 0.000001 is small, anyways stackoverflow.com/questions/51340559/… $\endgroup$
    – DuttaA
    Jul 31, 2018 at 0:46
  • $\begingroup$ you mean shallow as in I should add more layers to it? Or is there something I should change. $\endgroup$
    – WilsonPena
    Jul 31, 2018 at 0:48
  • 1
    $\begingroup$ shallow as in more layers to add, also if you use default tf.layers, it is quite difficult to set it up through estimators API, but it helps in fast convergence...maybe due to good weights initialisation $\endgroup$
    – DuttaA
    Jul 31, 2018 at 0:50

1 Answer 1


Two things:

  1. You're squashing the outputs via the sigmoid function before calculating the loss via tf.nn.sigmoid_cross_entropy_with_logits. This loss function takes the class logits as inputs, meaning you should be passing in the output from the linear layer without any non-linearity activation functions applied after. NOTE that applying a sigmoid function to the logits is a regularization technique used to diminish the effect of the model outputting very large logits, but in a simple three layer CNN like you've got going on here, using this technique is going to hurt more than help.

  2. You should be annealing the learning rate (rather than using a constant learning rate every epoch). As optimization starts to converge on a set of parameters, the parameter update magnitudes should get smaller and smaller otherwise you're very likely to jump past and around where you want to be.

Out of curiosity, what accuracy/loss are you getting?

  • $\begingroup$ I'm getting about 50% accuracy most of the time, but it varies a lot. The loss is around 9~12 most of the time. $\endgroup$
    – WilsonPena
    Jul 31, 2018 at 2:08
  • 1
    $\begingroup$ That's what I was imagining the accuracy to be. The loss is comparatively high; for a given example, the loss for each class is calculated as: max(x, 0) - x*z + log(1 + exp(-abs(x))), where x is the passed in logit for the class and z is the correct label (either 0 or 1). Given that you're sigmoid squashing the logits into the range of [0, 1], 12.5 is the highest loss possible, and you get a loss of ~9.25 if the model is just outputting sigmoid(x) = 0.5 for each class. Since you're still getting 50% accuracy, this means your model IS learning, and the sigmoid function applied to the output... $\endgroup$ Jul 31, 2018 at 2:37
  • 1
    $\begingroup$ ... IS regularizing the predictions as it's supposed to be (and as it's typically used). My guess is that if you decrease your initial learning rate to around 1e-4 and then anneal it exponentially (e.g. multiply it by 0.5 every epoch), then you'll see a lot of improvement in both the accuracy and loss. $\endgroup$ Jul 31, 2018 at 2:39
  • $\begingroup$ Thanks a LOT for the tips. I removed the first sigmoid and the cost started going down. Although it got stable and ~57% accuracy. But at least I'm not stuck at that point anymore. Thanks a lot! $\endgroup$
    – WilsonPena
    Jul 31, 2018 at 13:18
  • $\begingroup$ No problem! That's what SO is for :) It makes sense that you only saw a few percentage points increase in accuracy when removing that sigmoid activation since you're basically just removing one form of regularization from your model (as explained in my previous comment). Try using smaller and annealed learning rates and see how the training goes. $\endgroup$ Jul 31, 2018 at 21:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.