Problem with batching tensors - InvalidArgumentError: Cannot batch tensors with different shapes in component

Question

So, I am trying to build this model for an image classifier using the oxford flower dataset 102, and I am having issues when trying to fit the model. The error says:

InvalidArgumentError:  Cannot batch tensors with different shapes in component 0. First element had shape [500,667,3] and element 1 had shape [500,528,3].
 [[node IteratorGetNext (defined at <ipython-input-23-4e2ec1874986>:7) ]] [Op:__inference_train_function_36111]

Function call stack: train_function

I am struggling with this for quite some time but I really cannot fix it. I will post the code and relevant information below, so this is going to be a little bit, but the question is just about this error, you can skip till the end of the problem is obvious. Please, any light will help.

Note: there is an edit at the end ----

The code I am trying to run, is:

num_training_examples  = dataset_info.splits['train'].num_examples
num_val_examples = dataset_info.splits['validation'].num_examples
num_test_examples = dataset_info.splits['test'].num_examples
num_classes = dataset_info.features['label'].num_classes

print('There are {:,} images in the training set'.format(num_training_examples))
print('There are {:,} images in the validation set'.format(num_val_examples))
print('There are {:,} images in the test set'.format(num_test_examples))
print('\nThere are {:,} classes in our dataset'.format(num_classes))

Which outputs:

There are 1,020 images in the training set
There are 1,020 images in the validation set
There are 6,149 images in the test set
There are 102 classes in our dataset

Examples of 3 images:

print('Taking 3 images from the training set:')
for image, label in training_set.take(3):
    print('\n\u2022 dtype:', image.dtype, '\n\u2022 shape:', image.shape, '\n\u2022 label:', label)

with output (showing just one):

• dtype: <dtype: 'uint8'> 
• shape: (500, 667, 3) 
• label: tf.Tensor(72, shape=(), dtype=int64)

I created a pipeline:

image_size = (224, 224)
def normalize(image, label):
    normalized_image = tf.image.resize(image, image_size)
    normalized_image /= 255
    return normalized_image, label

batch_size = 32

training_batches = training_set.cache().shuffle(num_training_examples//4).batch(batch_size).map(normalize).prefetch(1)
validation_batches = validation_set.cache().shuffle(num_training_examples//4).batch(batch_size).map(normalize).prefetch(1)
testing_batches = test_set.cache().shuffle(num_training_examples//4).batch(batch_size).map(normalize).prefetch(1)

Now I build the classifier:

mobilenet_url = "https://tfhub.dev/google/tf2-preview/mobilenet_v2/feature_vector/4"

feature_extractor = hub.KerasLayer(mobilenet_url, input_shape= image_size + (3,), trainable=False)

layer_neurons = [512, 256, 256, 128, 100, 64, 32, 32]
dropout_rate = 0.3

model = tf.keras.Sequential([
        feature_extractor,
    ])

model.add(tf.keras.layers.Flatten(input_shape=(224,224,3)))

for neurons in layer_neurons:
    model.add(tf.keras.layers.Dense(neurons, activation = 'relu'))
    model.add(tf.keras.layers.Dropout(dropout_rate))

model.add(tf.keras.layers.Dense(10, activation = 'softmax'))
model.summary()

model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
    metrics=['accuracy'])

early_stopping = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)

save_best = tf.keras.callbacks.ModelCheckpoint('./best_model.h5',
monitor='val_loss',
save_best_only=True)

history = model.fit(training_batches,
epochs = 100,
validation_data=validation_batches,
callbacks=[early_stopping, save_best])

AND THE ERROR:

 Epoch 1/100

---

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-23-4e2ec1874986> in <module>
      5                         epochs = 100,
      6                         validation_data=validation_batches,
----> 7                         callbacks=[early_stopping, save_best])

And after several lines it says:

InvalidArgumentError:  Cannot batch tensors with different shapes in component 0. 
First element had shape [500,667,3] and element 1 had shape [500,528,3].
     [[node IteratorGetNext (defined at <ipython-input-23-4e2ec1874986>:7) ]] 
[Op:__inference_train_function_36111]

Function call stack:
train_function

EDIT: After changing the pipeline order:

batch_size = 32

training_batches = training_set.shuffle(num_training_examples//4).map(normalize).cache().batch(batch_size).prefetch(1)
validation_batches = validation_set.shuffle(num_training_examples//4).map(normalize).batch(batch_size).cache().prefetch(1)
testing_batches = test_set.shuffle(num_training_examples//4).map(normalize).batch(batch_size).cache().prefetch(1)

And keeping everything else the same, I did not got that error anymore, but a new error raised:

InvalidArgumentError                      Traceback (most recent call last)
<ipython-input-19-4e2ec1874986> in <module>
      5                         epochs = 100,
      6                         validation_data=validation_batches,
----> 7                         callbacks=[early_stopping, save_best])
...
...
...
InvalidArgumentError:  Received a label value of 95 which is outside the valid range of [0, 10).  Label values: 42 25 94 18 9 87 54 75 65 32 90 83 26 79 12 3 56 21 34 38 43 28 61 95 1 22 87 50 56 52 40 32
 [[node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits (defined at <ipython-input-19-4e2ec1874986>:7) ]] [Op:__inference_train_function_18131]

Function call stack:
train_function

Answer 1

Problem solved. It was a dumb and silly mistake after all. I was being naive - maybe I need to sleep, I don't know.

The problem was just the last layer of the network:

model.add(tf.keras.layers.Dense(10, activation = 'softmax'))

It was supposed to be

model.add(tf.keras.layers.Dense(num_classes, activation = 'softmax'))

I could not build a network with an argument of 10 restricting it to 10 outputs: I have 101 possible outputs!!!

Anyway, that's it. Sorry!