Is the output size of the last layer of a standard fully connected neural network the same as the input size?

Question

Let's say I have a neural net with Dense layers. The input layer has 3 neurons, the single hidden layer has 5 neurons, and the final output layer has 2 neurons.

For layer 1, 3 inputs go in and 5 inputs go out. For layer 2, the 5 inputs from layer 1 go in and 2 inputs go out. So for layer 3, would 2 inputs go in and 2 inputs come out?

Answer 1

In a Dense NN,

• Input to a Layer depends on the output of the previous layer and its Neuron count
• Previous Layer for the first layer is the Input Data Features
• Output of a Layer is equal to the Layer's neuron count. A copy of each goes to all the Neuron of the Next Layer

It means for your case,
- First Layer will have *M(Input features)3 inputs going in i.e. M to each Neuron and 3 coming out.
- The Second layer will have *3(Previous Layer)5 inputs going in and 5 coming out
- The Last layer will have 5(Previous Layer)*2 inputs going in and 2 coming out

A Neural Network is a complex Tensor operation. Arrow and Circle are logical representations. Each arrow got a weight and each Circle got the Activation function and the Bias term. So, you may count this way too.

$\hspace{3cm}$
Validate this with Keras Model.summary result which shows the parameter count.

Answer 2

No - the input and outputs sizes are independent from one another in a deep full-connected network. You could e.g. have input matrix shape (100, 100, 100) and output shape (1,).

Answer 3

I think you are confusing weights and neurons a bit.

Output of the last layer: problem-specific (binary: single neuron, multiclass/multilabel: number of neurons=number of Classes) number of neurons.

Input of the last layer: weights equal to the number of neurons in the previous layer + 1 (bias) per each output neuron.