Input 0 is incompatible with layer conv2d_2: expected ndim=4, found ndim=3 I get this error in Tensor flow, What does it mean and how can I fix it?

Question

import pickle 


import keras

from keras.models import Sequential

from keras.layers import Dense, Dropout, Flatten, Activation

from keras.layers import Conv2D, MaxPooling2D

from keras.utils import to_categorical

import numpy as np

X = pickle.load(open('cancer_image_features.pickle','rb'))

y = pickle.load(open('cancer_image_lables.pickle','rb'))


X = X/255.0

y = to_categorical(y)

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=X.shape[1:]))

model.add(Conv2D(64, (3, 3), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, kernel_size=(3, 3),activation='relu'))

model.add(Conv2D(64, (3, 3), activation='relu'))

model.add(MaxPooling2D(pool_size=(2, 2)))



model.add(Activation("softmax"))          
model.add (Flatten())

model.add(Dense(3))

model.compile ( loss = 'binary_crossentropy', optimizer = 'adam' , metrics = ['accuracy'])

model.fit(X,y, batch_size = 512, epochs = 10,  validation_split= 0.3)

Answer 1

The error is likely to come from this line:

model.add(Conv2D(32, kernel_size=(3, 3),activation='relu',input_shape=X.shape[1:]))

Input_shape should be a 4dim vectors as stated in the keras doc:

Input shape

4D tensor with shape: (batch, channels, rows, cols) if data_format is "channels_first" or 4D tensor with shape: (batch, rows, cols, channels) if data_format is "channels_last".

You may have to reshape your data, as stated here: https://stackoverflow.com/q/43895750/8119313

Answer 2

X.shape here as I guess is something similar to the mnist data, (60000, 28, 28), means it doesn't have extra dimension or say 24bit-representation, i.e., some color-bytes. As such, each x in X is having 2D shape, thus, X.shape[1:] -eq x.shape -eq (28, 28). You have to explicitly reshape X to include the extra dimension needed for Conv2D layer.

As per the code, seems you want to use 'channel_last' configuration, for which reshaping of X_train and X_test may go like:

X = X.reshape(list(X.shape) + [1])    # (60000, 28, 28, 1)

For 'channel_first' it will be:

X = X.reshape([X.shape[0], [1]] + list(X.shape[1:]))    # (60000, 1, 28, 28)

Hope it helps the reader.