The following code is to train a neural network model of a given dataset (50,000 samples, 64 dim).
from keras import Sequential from keras.layers import Dense from keras.optimizers import Adam X, y = process_dataset() model = Sequential([ Dense(16, input_dim=X.shape, activation='relu'), Dense(16, activation='relu'), Dense(1, activation='sigmoid') ]) ''' Compile the Model ''' model.compile(loss='binary_crossentropy', optimizer=Adam(lr=0.01), metrics=['accuracy']) ''' Fit the Model ''' model.fit(X, y, shuffle=True, epochs=1000, batch_size=200, validation_split=0.2, verbose=2)
In the beginning, you can see below that the
val_loss gets reduced from one epoch to another very well.
Epoch 82/1000 - 0s - loss: 0.2036 - acc: 0.9144 - val_loss: 0.2400 - val_acc: 0.8885 Epoch 83/1000 - 0s - loss: 0.2036 - acc: 0.9146 - val_loss: 0.2375 - val_acc: 0.8901
When the model takes many epochs, the loss change becomes so small, especially when the number of epochs increases.
Epoch 455/1000 - 0s - loss: 0.0903 - acc: 0.9630 - val_loss: 0.1317 - val_acc: 0.9417 Epoch 456/1000 - 0s - loss: 0.0913 - acc: 0.9628 - val_loss: 0.1329 - val_acc: 0.9443
Kindly, I have two questions:
- What does this phenomenon mean? i.e., the loss begins to decrease very well at the beginning but not much reduction by the time the training epochs takes a lot of iteration.
- What is the possible solution for this?