I want to train a convolutional neural network autoencoder on a csv file which contains values pixel neighborhood position of an original image of 1024x1024. When I try to train it, I have the following error that I don't manage to resolve. ValueError: Input 0 of layer max_pooling2d is incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 1, 1024, 1024, 16)
Do you have any idea what I am doing wrong ?
I managed to load my dataset, extract the x, y, and value columns as NumPy arrays and extract the relevant columns as NumPy arrays. My csv file contains 3 columns.
x = data[0].values
y = data[1].values
values = data[2].values
Then, I create an empty image with the correct dimensions and fill in the image with the pixel values
image = np.empty((1024, 1024))
for i, (xi, yi, value) in enumerate(zip(x, y, values)):
image[xi.astype(int), yi.astype(int)] = value
To use this array as input to my convolutional autoencoder I reshaped it to a 4D array with dimensions
# Reshape the image array to a 4D tensor
image = image.reshape((1, image.shape[0], image.shape[1], 1))
Finally, I declare the convolutional autoencoder structure, at this stage I have the error incompatible with the layer: expected ndim=4, found ndim=5. Full shape received: (None, 1, 1024, 1024, 16)
import keras
from keras.layers import Input, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
# Define the input layer
input_layer = Input(shape=(1,image.shape[1], image.shape[2], 1))
# Define the encoder layers
x = Conv2D(16, (3, 3), activation='relu', padding='same')(input_layer)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
encoded = MaxPooling2D((2, 2), padding='same')(x)
# Define the decoder layers
x = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation='relu')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)
# Define the autoencoder model
autoencoder = Model(input_layer, decoded)
# Compile the model
autoencoder.compile(optimizer='adam', loss='binary_crossentropy')
# Reshape the image array to a 4D tensor
image = image.reshape((1, image.shape[0], image.shape[1], 1))
# Train the model
autoencoder.fit(image, image, epochs=50, batch_size=1, shuffle=True)
I also tried also Reshape the image array to a 4D tensor image = image.reshape((1, image.shape[0], image.shape[1], 1))
but I have this error: ValueError: cannot reshape array of size 1048576 into shape (1,1,1024,1)
image.reshape((image.shape[0], image.shape[1], 1))
andInput(shape=(image.shape[0], image.shape[1], 1))
. $\endgroup$image.reshape
since at that time image.shape == (1, 1024, 1024, 1). I also addedpadding='same'
that I forgot on aConv2D
layer. It seems working now. Just I have a negative loss value during the training, I don't know if that good or bad. $\endgroup$