3
$\begingroup$

I am currently learning about Keras and have a problem with the input shape of a dense layer.I am currently trying to the mnist dataset.I understand that the input_shape for the train images is (60000,28,28) also i understand that keras ignores the first dimension as it is the batch size hence the input shape entered in the dense model should be (28,28) but when putting that i get an error yet when i put input shape of(784,) the model runs.Could someone please explain why is that so

(train_images, train_labels), (test_images, test_labels) = 
 mnist.load_data()

 print(train_images.shape)


 network = models.Sequential()
 network.add(layers.Dense(512, activation='relu', input_shape=(28,28)))
 network.add(layers.Dense(10, activation='softmax'))
$\endgroup$
  • $\begingroup$ What error do you get with shape (28,28)? When you use shape (784,), are you also flattening the input data (because without that I get an error there too)? $\endgroup$ – Ben Reiniger Feb 12 at 17:48
2
$\begingroup$

You always need to flatten your pictures when connecting the input to a Dense layer in Keras (Note that this is not the case for CNNs or RNNs). The reason is when the dense layer is built, based on the Dense layer code, the input dim is the last element you pass in the inputs (input_dim = input_shape[-1]). Therefore, although you are passing an input of (28,28) keras thinks that the shape is only 28. This also explains why the input of (,784) indeed works.

You can check the Dense layer code here

$\endgroup$
  • $\begingroup$ Hi TitoOrt, your answer is actually inaccurate, see mine. $\endgroup$ – Mark.F Feb 12 at 15:32
0
$\begingroup$

When using the Sequential model in Keras, you always have to provide shape of the input for the first layer (dense, convolutional, LSTM and whatever), as you can see in the official documentation:

"The model needs to know what input shape it should expect. For this reason, the first layer in a Sequential model (and only the first, because following layers can do automatic shape inference) needs to receive information about its input shape. There are several possible ways to do this."

By the way, in Keras, you don't actually need to flatten the layer prior to a dense layer, it is done automatically (see the note about dense layers from the official Keras documentation at the bottom).

"Note: if the input to the layer has a rank greater than 2, then it is flattened prior to the initial dot product with kernel." - Documentation.

$\endgroup$
  • $\begingroup$ But OP is providing the input shape...? Despite the documentation, when I try it I get an output shape from the first layer of (None, 28, 512). $\endgroup$ – Ben Reiniger Feb 12 at 17:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.