What kind of operation does Dense Layer perform to reduce dimemsion. So basically I have used Dense layer to compress the dimension all the time like from 10000 neurons to direct 2000 neurons or even 10 neurons for Output.

I'm not really able to understand what kind of operation does Dense layer perform to reduce the dimension from such higher number to lower number.


Let's stick with reducing 3 neurons to 2 neurons for simplicity (the mechanism will be the same for any number of neurons). Take the image below (taken from a StackOverflow post) as an example.

enter image description here.

Consider the transition from the second layer with 3 neurons to third layer with 2 neurons. All that happens is that the output of each of the 3 neurons of the 2nd layer is used as input to the each of the 2 neurons of the 3rd layer. To understand how this input-output mechanism works for each neuron, check this video.

  • $\begingroup$ so is it just the sum of previous layer neurons and there weights? $\endgroup$
    – Chris_007
    Oct 16 at 9:33
  • $\begingroup$ @Chris_007 not exactly, neurons should be considered as functions and output of the previous layer are the variables of this function. I modified the third link. That video should clarify things. $\endgroup$
    – serali
    Oct 16 at 9:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.