1
$\begingroup$

Im trying to build 2 simple networks with cleaned dataset for tweets sentiment classification(0/1):

  • one with all dense layers(binary bag of words)
  • another with RNN layer(embedding layer). But it both cases the validation loss and accuracy are always low. Pasting the code and graphs for reference.
    keras.layers.Dense(250, input_shape=(500,),activation='relu'),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(250, activation='relu'),
    keras.layers.Dropout(0.2),
    keras.layers.Dense(1, activation='sigmoid')
])

model2 = keras.models.Sequential([
            keras.layers.Embedding(input_dim=vocabulary,
                     output_dim=EMBEDDING_DIM,
                     input_length=max_length,
                     mask_zero=True),
            keras.layers.SimpleRNN(500, activation='relu'),
            keras.layers.Dropout(0.5),
            keras.layers.Dense(100, activation='relu'),
            keras.layers.Dropout(0.5),
            keras.layers.Dense(100, activation='relu'),
            keras.layers.Dropout(0.5),
            keras.layers.Dense(1, activation='sigmoid')           
])
[![enter image description here][1]][1]```

Tried increasing network complexity by adding more layers and added dropouts. Still nothing increase the loss and accuracy. What am i missing?


  [1]: https://i.sstatic.net/tKlYS.png
$\endgroup$

1 Answer 1

3
$\begingroup$

You are experiencing a lot of overfitting on the training set here.

I would go back and see if there are any inherent issues with the data (scaling? class imbalance? etc.) before diving into modeling.

Also, be aware that if you are using built-in validation (e.g. validation_split=0.3) in TF/Keras, it does not shuffle the data or randomly sample and so if your data are sorted by class label this can seriously impact your results.

$\endgroup$
1
  • 1
    $\begingroup$ Thank you scaling was the issue, used minmax scaler to normalize he vectors and was able to get 60% accracy in both training and validation set $\endgroup$
    – emily
    Commented Sep 2, 2023 at 17:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.