How to correct ValueError about incompatible dimensions of training data set in locallyconnected1D layer NN

Question

I'm training this simple network with a few points, but it can't train. The model looks okay, but when training it raises a ValueError about the dimension of the training data sets. Could someone help? Thanks in advance.

The code is;

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, LocallyConnected1D

# Generate random training data
X_train = np.random.rand(1000, 100, 1)
y_train = np.random.randint(0, 10, size=(1000, 1))

# Generate random testing data
X_test = np.random.rand(100, 100, 1)
y_test = np.random.randint(0, 10, size=(100, 1))

# Create a sequential model
model = Sequential()

# Add a LocallyConnected1D layer with 32 filters, a kernel size of 3, and a ReLU activation function
model.add(LocallyConnected1D(32, 3, activation='relu', input_shape=(100, 1)))

# Add a dense layer with 10 units and a softmax activation function
model.add(Dense(10, activation='softmax'))

# Compile the model with categorical cross-entropy loss and Adam optimizer
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

# Train the model on the training data
model.fit(X_train, y_train, epochs=10, batch_size=32, validation_data=(X_test, y_test))

# Evaluate the model on the testing data
test_loss, test_acc = model.evaluate(X_test, y_test)
print('Test loss:', test_loss)
print('Test accuracy:', test_acc)

and the ValueError raised is:

ValueError: in user code:

    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1160, in train_function  *
        return step_function(self, iterator)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1146, in step_function  **
        outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1135, in run_step  **
        outputs = model.train_step(data)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 994, in train_step
        loss = self.compute_loss(x, y, y_pred, sample_weight)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\training.py", line 1052, in compute_loss
        return self.compiled_loss(
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\engine\compile_utils.py", line 265, in __call__
        loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\losses.py", line 152, in __call__
        losses = call_fn(y_true, y_pred)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\losses.py", line 272, in call  **
        return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\losses.py", line 1990, in categorical_crossentropy
        return backend.categorical_crossentropy(
    File "C:\ProgramData\Anaconda3\envs\tf\lib\site-packages\keras\backend.py", line 5529, in categorical_crossentropy
        target.shape.assert_is_compatible_with(output.shape)

    ValueError: Shapes (None, 1) and (None, 98, 10) are incompatible

Answer 1

To elaborate on Ubikuity's comment, there are a couple of issues. The error that you are seeing is because your model output shape is (None, 98, 10), so it will output a 2d tensor. The y_train shape that it is trying to compare against is (1000,1), so they are incompatible. You can specify y_train and y_test to be 1d by simply omitting the last size dimension:

y_train = np.random.randint(0, 10, size=(1000, ))
y_test = np.random.randint(0, 10, size=(100, ))

The second issue is that the output of your LocallyConnected1D layer fed into a Dense layer will have an output of shape (None, 98, 10). An easy fix is to flatten the output of the LocallyConnected1D layer before the Dense layer:

model.add(Flatten())

Then your model output shape will be (None, 10), so you can either add a final Dense layer with a single output, or else change your existing final Dense layer to a single output, or reshape your targets to be size=(100,10).

model.add(Dense(10, activation='softmax'))
model.add(Dense(1, activation='softmax'))

or else instead of 1D inputs previously mentioned, do something like :

y_train = np.random.randint(0, 10, size=(1000, 10))
y_test = np.random.randint(0, 10, size=(100, 10))