1
$\begingroup$

I am working on a sentiment analysis problem which is a binary classification. These are some of the parameters that might be useful:

1.) Length of train list = 203

2.) Length of test list = 51

3.) vocabulary size = 20,000

4.) sentence length = 35

5.) dimension of embedding layer = 50

First I One hot encode the train and test in order to encode the text data into integers. Then I use pad_sequences to convert all texts into same length(35). Then I use an Embedding layer to convert my text into vector representation with dimension of 50, and then apply a LSTM layer and finally a Dense layer with sigmoid activation function. This is the code:

# convert into one hot representation
ohr_train = [one_hot(i, vocab_size) for i in train_x2]
ohr_test = [one_hot(i, vocab_size) for i in test_x2]

# pad the text
sent_len = 35
train_embedded_docs = pad_sequences(ohr_train, padding = 'pre', maxlen = sent_len)
test_embedded_docs = pad_sequences(ohr_test, padding = 'pre', maxlen = sent_len)

# model architecture
dimension = 50
model = Sequential()
model.add(Embedding(vocab_size, dimension, input_length = 35))
model.add(LSTM(units = 50, activation = 'leaky_relu', kernel_initializer = 'he_uniform'))
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid', kernel_initializer = 'glorot_uniform'))
model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
print(model.summary())

# converting to array
X_train_final = np.array(train_embedded_docs)
X_test_final = np.array(test_embedded_docs)
y_train_final = np.array(train_y1)
y_test_final = np.array(test_y1)

#fit and evaluate the model
model.fit(X_train_final, y_train_final, validation_split = 0.2, batch_size=32, epochs = 10)
model.evaluate(X_test_final, y_test_final)

Now this works fine but when I try to tune the HP using KerasTuner, then I get an error as ValueError: Exception encountered when calling layer sequential (type Sequential). Input 0 of layer "lstm" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 35)

Call arguments received:
  • inputs=tf.Tensor(shape=(None, 35), dtype=int32)
  • training=True
  • mask=None`

This is my KerasTuner function:

# TUNE THE NUMBER OF LAYERS, NEURONS, LEARNING RATE OF THE NEURAL NET

def build_model(hp):
  model = Sequential()

  for i in range(hp.Int('LSTM Layers', min_value = 1, max_value = 3)):
    model.add(LSTM(hp.Int('units', min_value = 32, max_value = 512, step = 32)))
    
    
  model.add(Dense(units = 1, activation = 'sigmoid', kernel_initializer='glorot_uniform'))

  model.compile(
      optimizer=tf.keras.optimizers.Adam(learning_rate=hp.Float('lr', min_value = 1e-4, max_value = 1e-2, sampling = 'log')), 
      loss = 'binary_crossentropy', metrics = ['accuracy'])

  return model

tuner = kt.RandomSearch(
    build_model,
    kt.Objective('val_loss', direction = 'min'),
    max_trials=30, seed = 69)

tuner.search(X_train_final, y_train_final, epochs=5, batch_size = 32, validation_data=(X_test_final, y_test_final))

Why is it asking for 3 dimensions for tuning when it worked perfectly fine for training?

$\endgroup$

1 Answer 1

1
$\begingroup$

I finally got where the error was coming from. The LSTM layer takes in an input of 3 dimensions from the Embedding layer as can be seen from the code above. But in my KerasTuner function, I had forgotten to include an Embedding layer before the LSTM layer which was the reason I was getting the above mentioned error!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.