Predictions of a Deep Learning Model

Question

I’ve built a model that classifies digits from 0-9. My dataset is tf.keras.datasets.mnist. I use softmax as the activation function for the output layer.

Q1:

The output layer should consist of 10 neurons. Each representing a digit from 0-9. But even when I change the number of output layer neurons to 20, the predictions are accurate. Seems as if even though I only need to predict 10 digits, I can have more than 10 neurons in the output layer. Why’s that?

Q2:

Also, the prediction is a list of lists right. Each list containing probabilities corresponding to 10 neurons and we take the neuron having the highest probability. My question is, suppose the 6th neuron has the highest probability, how do I know what digit(label) is assigned to it?

I’m a complete newbie to deep learning, so please dumb it down for me.

Answer 1

Q1: I do not really understand how the situation in your Q1 is possible - I would expect an error to be thrown about as a mismatch in shape. For example, when I change the number of classes in the final dense layer, I do indeed get an error.

model = Sequential()
model.add(Conv2D(32, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=input_shape))
model.add(Conv2D(64, (3, 3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
# adjust the number of layers from 10 to 20 - which throws an error
model.add(Dense(20, activation='softmax'))
model.compile(loss= 'categorical_crossentropy',
          optimizer = 'rmsprop',
          metrics=['accuracy'])

Output:

ValueError: A target array with shape (60000, 10) was passed for an output of shape (None, 20) while using as loss categorical_crossentropy. This loss expects targets to have the same shape as the output.

Perhaps you can post your code when you build in the layers.

Q2: The list of probabilities that are output by the model will match the order of the classes on which you trained it. For example, if the order of the classes for the digits 0-9 is [0, 1, 2, 3, ... 9] and the model outputs [0.91, 0.01... 0.01] for a certain image, then the model thinks there is a 91% chance that the image is a '0'