I have trained a 4 layer neural network

model = Sequential()

#get number of columns in training data
n_cols = X_train.shape[1]

#add model layers
model.add(Dense(8, activation='relu', input_shape=(n_cols,)))
model.add(Dense(8, activation='relu'))
model.add(Dropout(rate = 0.05))
model.add(Dense(8, activation='relu'))

#adam = optimizers.Adam(lr=0.0001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)
model.compile(optimizer='adam', loss='mae')
history = model.fit(X_train, y_train, epochs= 200, validation_split=0.2, batch_size=128)

When I plot the graph between train and validation loss the graph seems to be like enter image description here The validation loss is fluctuating. Am I doing it right?


Overfitting is a situation when model starts to perform more better on training set than on validation (example of such behaviour: loss curves are moving to different ways). According to your plot the model hasn't overfitted. Validation loss seems to fluctuating more than train, because you have more points in training dataset and errors on test have higher influence while loss is calculated.

  • $\begingroup$ So, do you mean to say the fluctuation of validation loss is normal here? $\endgroup$ – Bhaskar Dhariyal Aug 20 '19 at 7:00
  • 1
    $\begingroup$ Yes, it's normal behaviour. $\endgroup$ – Lana Aug 20 '19 at 7:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.