Dimension error when tuning LSTM layer

Question

I am working on a sentiment analysis problem which is a binary classification. These are some of the parameters that might be useful:

1.) Length of train list = 203

2.) Length of test list = 51

3.) vocabulary size = 20,000

4.) sentence length = 35

5.) dimension of embedding layer = 50

First I One hot encode the train and test in order to encode the text data into integers. Then I use pad_sequences to convert all texts into same length(35). Then I use an Embedding layer to convert my text into vector representation with dimension of 50, and then apply a LSTM layer and finally a Dense layer with sigmoid activation function. This is the code:

# convert into one hot representation
ohr_train = [one_hot(i, vocab_size) for i in train_x2]
ohr_test = [one_hot(i, vocab_size) for i in test_x2]

# pad the text
sent_len = 35
train_embedded_docs = pad_sequences(ohr_train, padding = 'pre', maxlen = sent_len)
test_embedded_docs = pad_sequences(ohr_test, padding = 'pre', maxlen = sent_len)

# model architecture
dimension = 50
model = Sequential()
model.add(Embedding(vocab_size, dimension, input_length = 35))
model.add(LSTM(units = 50, activation = 'leaky_relu', kernel_initializer = 'he_uniform'))
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid', kernel_initializer = 'glorot_uniform'))
model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
print(model.summary())

# converting to array
X_train_final = np.array(train_embedded_docs)
X_test_final = np.array(test_embedded_docs)
y_train_final = np.array(train_y1)
y_test_final = np.array(test_y1)

#fit and evaluate the model
model.fit(X_train_final, y_train_final, validation_split = 0.2, batch_size=32, epochs = 10)
model.evaluate(X_test_final, y_test_final)

Now this works fine but when I try to tune the HP using KerasTuner, then I get an error as ValueError: Exception encountered when calling layer sequential (type Sequential). Input 0 of layer "lstm" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 35)

Call arguments received:
  • inputs=tf.Tensor(shape=(None, 35), dtype=int32)
  • training=True
  • mask=None`

This is my KerasTuner function:

# TUNE THE NUMBER OF LAYERS, NEURONS, LEARNING RATE OF THE NEURAL NET

def build_model(hp):
  model = Sequential()

  for i in range(hp.Int('LSTM Layers', min_value = 1, max_value = 3)):
    model.add(LSTM(hp.Int('units', min_value = 32, max_value = 512, step = 32)))
    
    
  model.add(Dense(units = 1, activation = 'sigmoid', kernel_initializer='glorot_uniform'))

  model.compile(
      optimizer=tf.keras.optimizers.Adam(learning_rate=hp.Float('lr', min_value = 1e-4, max_value = 1e-2, sampling = 'log')), 
      loss = 'binary_crossentropy', metrics = ['accuracy'])

  return model

tuner = kt.RandomSearch(
    build_model,
    kt.Objective('val_loss', direction = 'min'),
    max_trials=30, seed = 69)

tuner.search(X_train_final, y_train_final, epochs=5, batch_size = 32, validation_data=(X_test_final, y_test_final))

Why is it asking for 3 dimensions for tuning when it worked perfectly fine for training?

Answer 1

I finally got where the error was coming from. The LSTM layer takes in an input of 3 dimensions from the Embedding layer as can be seen from the code above. But in my KerasTuner function, I had forgotten to include an Embedding layer before the LSTM layer which was the reason I was getting the above mentioned error!