0
$\begingroup$

Can anyone explain why we need to flatten the data before inputting it into a fully-connected layer? What will happen if we input a matrix of size (m,n) into a fully-connected layer that has k activation units?

I recently tried to input a matrix of X (126976,4) to a DENSE layer with 255 neurons and got the weight matrix of (4,255), which I can't explain why? My training output Y is the size (126976,255).

$\endgroup$

1 Answer 1

0
$\begingroup$

To answer your question:

  • In a fully connected layer, also known as a dense layer, each neuron is connected to every neuron in the previous layer. This means that each neuron receives an input from all the neurons in the previous layer.

  • When you input a matrix of size (m,n) into a fully-connected layer, the layer expects a 1D array as input. This is because the weights of the fully connected layer are also a 1D array. If you input a 2D array (or matrix), the layer will not be able to correctly compute the dot product between the input and the weights.

  • Flattening the data converts the 2D array into a 1D array, which can then be correctly processed by the fully connected layer.

  • In your case, you input a matrix of size (126976,4) into a dense layer with 255 neurons. The weight matrix of size (4,255) is because each of the 4 input features is connected to each of the 255 neurons in the dense layer, resulting in a total of 4*255 = 1020 connections.

  • The output size of (126976,255) is because for each of the 126976 samples, there are 255 neurons in the dense layer, each producing an output.

Hope I answered your question.

$\endgroup$
2
  • $\begingroup$ Thank you so much. I now understand the Dense Layer more!!! $\endgroup$ Commented Jul 6, 2023 at 5:20
  • $\begingroup$ Happy to help you! $\endgroup$ Commented Jul 6, 2023 at 5:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.