help with Keras sequential model output

Question

I have trained a sequential model with keras for MNIST dataset and this is the code I've used.

# Create the model
model = Sequential()

# Add the first hidden layer
model.add(Dense(50, activation='relu', input_shape = (X.shape[1],)))

# Add the second hidden layer
model.add(Dense(50, activation='relu'))

# Add the output layer
model.add(Dense(10, activation = 'softmax'))

# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics = ['accuracy'])

# Fit the model
model.fit(X, y, validation_split=.3)

Output:

Train on 1750 samples, validate on 750 samples
1750/1750 [==============================] - 0s - loss: 0.1002 - acc: 0.9811 - val_loss: 0.3777 - val_acc: 0.8800

Can you explain what is loss, acc , val_loss, val_acc? How can I know my model performance from these metrics in output. Please explain if possible.

Answer 1

First will explain the terms loss, acc, val_loss, val_acc and then get into evaluating model performance.

Loss - This metric is used to understand the prediction by the network on training data vs the actual value. Given that you are using categorical crossentropy as your loss mechanism, your loss is actually telling you how near or far your predicitions are from the true classification values.

Acc - Is the accuracy of the network on the training data. After training on 1750 samples, your network can accurately predict with 0.98 accuracy.

Val_loss, Val_acc - The metrics are the same but are evaluated on validation data, that is the data that is not part of the training dataset, this will help you evaluate how your network performs on data that it is not trained on, sort of mimicking how your network would work when deployed in the real world on the chosen dataset.

The above metrics help you understand how your network is performing every epoch, giving you an idea on how to improve the performance.

How to perform model performance:

Model performance requires you to create/manage training dataset that would depict what you can expect in the real world when deploying your model to actually predict.

You would look at test accuracy, precision, recall, f1 scores, confusion matrix and other metrics based on the problem you are solving to truly evaluate your model performance.

Hope this answers your question. Have fun with deep learning networks :)