I am working on a sentiment analysis problem which is a binary classification. These are some of the parameters that might be useful:
1.) Length of train list = 203
2.) Length of test list = 51
3.) vocabulary size = 20,000
4.) sentence length = 35
5.) dimension of embedding layer = 50
First I One hot encode the train and test in order to encode the text data into integers. Then I use pad_sequences
to convert all texts into same length(35). Then I use an Embedding
layer to convert my text into vector representation with dimension of 50, and then apply a LSTM
layer and finally a Dense
layer with sigmoid
activation function. This is the code:
# convert into one hot representation
ohr_train = [one_hot(i, vocab_size) for i in train_x2]
ohr_test = [one_hot(i, vocab_size) for i in test_x2]
# pad the text
sent_len = 35
train_embedded_docs = pad_sequences(ohr_train, padding = 'pre', maxlen = sent_len)
test_embedded_docs = pad_sequences(ohr_test, padding = 'pre', maxlen = sent_len)
# model architecture
dimension = 50
model = Sequential()
model.add(Embedding(vocab_size, dimension, input_length = 35))
model.add(LSTM(units = 50, activation = 'leaky_relu', kernel_initializer = 'he_uniform'))
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid', kernel_initializer = 'glorot_uniform'))
model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])
print(model.summary())
# converting to array
X_train_final = np.array(train_embedded_docs)
X_test_final = np.array(test_embedded_docs)
y_train_final = np.array(train_y1)
y_test_final = np.array(test_y1)
#fit and evaluate the model
model.fit(X_train_final, y_train_final, validation_split = 0.2, batch_size=32, epochs = 10)
model.evaluate(X_test_final, y_test_final)
Now this works fine but when I try to tune the HP using KerasTuner
, then I get an error as ValueError: Exception encountered when calling layer sequential (type Sequential). Input 0 of layer "lstm" is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 35)
Call arguments received:
• inputs=tf.Tensor(shape=(None, 35), dtype=int32)
• training=True
• mask=None`
This is my KerasTuner
function:
# TUNE THE NUMBER OF LAYERS, NEURONS, LEARNING RATE OF THE NEURAL NET
def build_model(hp):
model = Sequential()
for i in range(hp.Int('LSTM Layers', min_value = 1, max_value = 3)):
model.add(LSTM(hp.Int('units', min_value = 32, max_value = 512, step = 32)))
model.add(Dense(units = 1, activation = 'sigmoid', kernel_initializer='glorot_uniform'))
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=hp.Float('lr', min_value = 1e-4, max_value = 1e-2, sampling = 'log')),
loss = 'binary_crossentropy', metrics = ['accuracy'])
return model
tuner = kt.RandomSearch(
build_model,
kt.Objective('val_loss', direction = 'min'),
max_trials=30, seed = 69)
tuner.search(X_train_final, y_train_final, epochs=5, batch_size = 32, validation_data=(X_test_final, y_test_final))
Why is it asking for 3 dimensions for tuning when it worked perfectly fine for training?