The following code is to train a neural network model of a given dataset (50,000 samples, 64 dim).
from keras import Sequential
from keras.layers import Dense
from keras.optimizers import Adam
X, y = process_dataset()
model = Sequential([
Dense(16, input_dim=X.shape[1], activation='relu'),
Dense(16, activation='relu'),
Dense(1, activation='sigmoid')
])
'''
Compile the Model
'''
model.compile(loss='binary_crossentropy', optimizer=Adam(lr=0.01), metrics=['accuracy'])
'''
Fit the Model
'''
model.fit(X, y, shuffle=True, epochs=1000, batch_size=200, validation_split=0.2, verbose=2)
In the beginning, you can see below that the val_loss
gets reduced from one epoch to another very well.
Epoch 82/1000
- 0s - loss: 0.2036 - acc: 0.9144 - val_loss: 0.2400 - val_acc: 0.8885
Epoch 83/1000
- 0s - loss: 0.2036 - acc: 0.9146 - val_loss: 0.2375 - val_acc: 0.8901
When the model takes many epochs, the loss change becomes so small, especially when the number of epochs increases.
Epoch 455/1000
- 0s - loss: 0.0903 - acc: 0.9630 - val_loss: 0.1317 - val_acc: 0.9417
Epoch 456/1000
- 0s - loss: 0.0913 - acc: 0.9628 - val_loss: 0.1329 - val_acc: 0.9443
Kindly, I have two questions:
- What does this phenomenon mean? i.e., the loss begins to decrease very well at the beginning but not much reduction by the time the training epochs takes a lot of iteration.
- What is the possible solution for this?
Thank you,