1
$\begingroup$

When we train a neural network model for a classification problem, we usually have a dense output layer of size equal to the number of labels we have.

If the layer size was greater, the model can still be trained and have output

How can we interpret such output? are there any applications for this?

$\endgroup$
  • $\begingroup$ Are you looking for a solution for the target which is not yet introduced but probably will come in furlturw? $\endgroup$ – vipin bansal Jul 16 at 13:38
  • $\begingroup$ @vipinbansal not really $\endgroup$ – Abdulrahman Bres Jul 16 at 13:58
3
$\begingroup$

The output layer is usually the same size as the last dense layer because we apply a loss function to train the model by comparing the last layer to what the output should be. If your output layer was bigger, it's less intuitive what your loss function should be, but the interpretation of your output would likely come from how you define this loss.

$\endgroup$
1
$\begingroup$

The interpretation of the output depends not only on the architecture of the network, but also on the final-layer activation functions and the training procedure. Most importantly, training a neural net requires you to choose a loss function, which describes how far off the predictions in the final layer are from ground truth. If you can specify a sensible loss function, then you've implicitly defined how the output layer is to be interpreted.

Offhand, I can't think of a classification problem where it would be helpful to have more output neurons than labels (but maybe someone else is more creative!)

One kind-of similar case are the advantage actor-critic networks commonly used in reinforcement learning. The ultimate goal of a reinforcement-learning agent is to choose an action from a set of $n$ possible actions, so traditionally we might try a network with $n$ outputs. Actor critic methods actually have $n+1$ outputs. The first $n$ choose an action, and the "extra" neuron tries to estimate the value of the chosen action.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.