I am trying the implement the CNN architecture introduced in Srivastava et al. 2014 Dropout paper (appendix B.2), for the SVHN dataset. I implemented only the convolutional layers part, without dropout or any regularization for the moment, following this description of the mentioned paper: "The convolutional layers have 96, 128 and 256 filters respectively. Each convolutional layer has a 5 × 5 receptive field applied with a stride of 1 pixel. Each max pooling layer pools 3 × 3 regions at strides of 2 pixels."

This is my code, implemented in TensorFlow 2.0 with Keras API

    from tensorflow.keras import layers, Sequential

    model = Sequential(name= "fMap_svhn_DANN")

    model.add(layers.Conv2D(filters= 96, kernel_size= 5, activation= 'relu', input_shape= (32,32,3)))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))

    model.add(layers.Conv2D(filters=128, kernel_size= 5, activation= 'relu'))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))

    model.add(layers.Conv2D(filters= 256, kernel_size= 5, activation= 'relu'))
    model.add(layers.MaxPool2D(pool_size= 3, strides= 2))


And this is the error I get"

Negative dimension size caused by subtracting 5 from 4 for 'conv2d_11/Conv2D' (op: 'Conv2D') with input shapes: [?,4,4,128], [5,5,128,256].

Any idea to help me ?


1 Answer 1


This means that your input shape was too small by the time it reached the last Conv2D layer.

By removing the last Conv2D and MaxPool2D layer, this is how the model looks like: enter image description here

You can see that the last output before the Flatten layer has height and width of 4, which is smaller than your kernel_size of 5 in the last Conv2D layer.

Heres what you can do:

  1. Increase the input image height and width OR
  2. Set padding='same' in the Conv2D layers
  • $\begingroup$ I cannot change the input dimensions as they were used in the mentioned paper, so I will go for option 2. $\endgroup$ May 9, 2020 at 7:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.