0
$\begingroup$

Input: a tensorflow dataset with 17000 items:

<_TensorSliceDataset element_spec=(TensorSpec(shape=(2000, 8), dtype=tf.float32, name=None), TensorSpec(shape=(6,), dtype=tf.float32, name=None))>

I receive errors, when I want to train a model. The definition with the model starts with

# INPUT 
    inp = tf.keras.Input(shape=(2_000,8))

when I call

history = model.fit(dataset, verbose=VERBOSE,
                batch_size=32,
              epochs=1)

The Error comes:

ValueError: Input 0 of layer "model_1" is incompatible with the layer: expected shape=(None, 2000, 8), found shape=(2000, 8)

What is the difference between (None, 2000, 8) and (2000, 8)? Why do these two not fit together?

How would I tune the model to expect the shape (2000, 8) ? What would I have to do to the dataset to deliver the shape (None, 2000, 8) ?

Here is some code to reproduce the error:

import keras
import tensorflow as tf
import numpy as np

def create_dataset(items=100, features=8, datapoints=20, classes = 6):
   xdata = np.zeros((items, datapoints, features), dtype="float16")
   ydata = np.zeros((items, classes), dtype="float16")
   for i in range(items):
       xdata[i, :, :] = np.random.rand(datapoints, features)
       ydata[i, :] = np.random.rand(classes)
       
   return tf.data.Dataset.from_tensor_slices((xdata, ydata))

ds = create_dataset()

def in_block(x, filters, kernel_size):
   return tf.keras.layers.Conv1D(filters = filters,
              kernel_size = 1,
              padding = 'same')(x)
   
def build_toy_model(datapoints, features, classes):
       
   # INPUT 
   inp = tf.keras.Input(shape=(datapoints,features))
   
   ############
   # FEATURE EXTRACTION SUB MODEL
   inp2 = tf.keras.Input(shape=(datapoints,1))
   x = in_block(inp2, 8, 3)
   x = in_block(x, 16, 3)
   model2 = tf.keras.Model(inputs=inp2, outputs=x)
   ###########
   
   all_chains = [model2(inp[:, :, i]) for i in range(features)]
   
   # COMBINE CHAINS
   y = tf.keras.layers.Concatenate()(all_chains)
   y = tf.keras.layers.Dense(64,activation='softmax', dtype='float32')(y)
   y = tf.keras.layers.Dense(classes,activation='softmax', dtype='float32')(y)
   
   # COMPILE MODEL
   model = tf.keras.Model(inputs=inp, outputs=y)
   opt = tf.keras.optimizers.Adam(learning_rate = 1e-3)
   loss = tf.keras.losses.KLDivergence()
   model.compile(loss=loss, optimizer = opt)
   
   return model

model = build_toy_model(features=8, datapoints=20, classes = 6)

model.fit(ds)

Edit: See Kaggle

$\endgroup$

2 Answers 2

1
$\begingroup$

In short. This error disappears if you call

model.fit(ds.batch(1))

See this post: How to use tensorflow dataset to fit a tf.keras model? why do I need to batch?

$\endgroup$
1
  • $\begingroup$ Links may not last forever, can you summarize the point you want the link to make in your answer? $\endgroup$
    – m13op22
    Feb 27 at 17:46
0
$\begingroup$

(None, 2000, 8) has an extra dimension, with the first dimension being the number of samples/observations. As can be seen in the documentation, the shape argument of tf.keras.Input needs to be the shape of the data excluding the batch size. In your case, this would be (8,).

$\endgroup$
2
  • $\begingroup$ so I have 17000 observation and each observation has the format (2000, 8). The Input of the Neural net also should have (2000, 8). I think that it relates to how to put the dataset into the fit method. In general it should work. It did work via a Simple Pipeline. But I am not happy with the documentation of those... tensorflow.org/ranking/api_docs/python/tfr/keras/pipeline/… $\endgroup$
    – Fabi
    Feb 15 at 4:20
  • $\begingroup$ As this answer on stackoverflow mentions, you need to get a batch of data from the dataset which you then pass to fit. $\endgroup$
    – Oxbowerce
    Feb 15 at 17:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.