I'm training a
Bidirectional LSTM using
My task is to predict the words order in a sentence, so, given a sentence, output of each timestep will be a real number: predicted real numbers of the sentence are ranked in order to obtain integer numbers, indicating the predicted position of the word in the sentence.
sentence -> "Nice I am"
predicted real numbers -> [0.2, 0.6, 0.4]
ranked real numbers -> [3,1,2]
Basically, I pad my sequences to 20, that is the max sequence length found in the dataset.
My model is the following:
model = tf.keras.Sequential() model.add(Masking(mask_value=0., input_shape=(timesteps, features))) model.add(Bidirectional(LSTM(units=timesteps, return_sequences=True), input_shape=(timesteps, features))) model.add(Dropout(0.2)) model.add(Dense(1, activation='linear'))
I mask all the padded vectors to avoid noise in learning. My training loss and my validation loss, during the training, are both low and around 0.33.
In the prediction phase, however, I obtain very bad results. I don't think my model is overfitting/underfitting looking at losses trend.
Is there something bad in my model architecture?
Thanks in advance.