4
$\begingroup$

I am training a model that get accuracy on train set up to 99%
but the validation split test not increase more than 70-72%

this is how my model is configurated:

model = Sequential()
#model.add(TimeDistributed(Dense(64, activation='linear')))
#model.add(GRU(512, return_sequences=True, activation='linear',))
model.add(LSTM(128, return_sequences=True, activation='linear'))
model.add(Conv1D(64, 64, strides=1, padding='same'))
model.add(Dense(128, activation="linear"))#, kernel_regularizer=regularizers.l1_l2(l1=1e-4, l2=1e-6)
model.add(Dropout(0.2))


#model.add(Conv1D(16, 16, strides=1, padding='same'))
#model.add(TimeDistributed(Dense(32, activation='linear')))
#model.add(GRU(512, return_sequences=True, activation='linear'))
model.add(LSTM(128, return_sequences=True, activation='linear'))
model.add(Conv1D(64, 64, strides=1, padding='same'))
model.add(Dense(128, activation="linear"))#,kernel_regularizer=regularizers.l1_l2(l1=1e-4, l2=1e-6)
model.add(Dropout(0.2))
#model.add(BatchNormalization())

model.add(Dense(1,activation='linear'))

model.compile(optimizer='adamax',loss="mse",metrics=['accuracy'])
model.fit(X_train,y_train,epochs=8000, batch_size=256, verbose=1, validation_split=0.1, callbacks=callback)

what can be the issue?

$\endgroup$
1
  • 1
    $\begingroup$ I would highly recommend you check it out visually, it becomes really obvious what's going on when you can literally see the way an overtrained model delineates. $\endgroup$ – stevec Apr 30 at 12:39
10
$\begingroup$

This means your model is overfitting the train set. Try simplifying the model by reducing the number of neurons in each layer. That will reduce your train accuracy but may allow your validation accuracy to increase.

You know that you are not overfitting when both train and validation accuracy hover around the same value.

$\endgroup$
2
  • 1
    $\begingroup$ Just to make sure our poor OP is as confused as possible, I'd like to add that increasing the number of neurons might also do the trick :) arxiv.org/abs/1811.04918 $\endgroup$ – John Madden Apr 30 at 13:50
  • $\begingroup$ lol true but I would say definite overfitting is taking place $\endgroup$ – MCP_infiltrator May 6 at 4:42
4
$\begingroup$

Your model is probably overfitting. You could try following things (or mixture of these):

  • Remove layers or reduce the number of neurons
  • Use dropout technique
  • Use regularization (e.g L1,L2)
  • Use Data Augmentation

copied from: https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.