Skip to main content
Data Science

Return to Answer

Reformatted, embedded image
Source Link
edit approved Aug 9, 2017 at 12:00
tuomastik
edit approved Aug 9, 2017 at 12:00
tuomastik
  • 1.2k
  • 10
  • 22

@answeringAnswering to @Media's comment about why there are many local minimasminima: http://www.chm.bris.ac.uk/pt/allan/elevels.gif

enter image description here

Consider, your randomly initialized weights to be belonging to energy level E4 in above image. Clearly, E1, E2, E3 have lesser energy. So all these 3 levels are minima w.r.t. E4. Now, with neural networks  ,your your learning algorithms is such that it is guaranteed[yes [yes, mathematical guarantee] to take you to a lesser energy level. The trouble is you don't know which lower state it will put you into. But, oh, lower, it will be!

@answering to @Media's comment about why there are many local minimas: http://www.chm.bris.ac.uk/pt/allan/elevels.gif

Consider, your randomly initialized weights to be belonging to energy level E4 in above image. Clearly, E1, E2, E3 have lesser energy. So all these 3 levels are minima w.r.t. E4. Now, with neural networks  ,your learning algorithms is such that it is guaranteed[yes, mathematical guarantee] to take you to a lesser energy level. The trouble is you don't know which lower state it will put you into. But, oh, lower, it will!

Answering to @Media's comment about why there are many local minima:

enter image description here

Consider, your randomly initialized weights to be belonging to energy level E4 in above image. Clearly, E1, E2, E3 have lesser energy. So all these 3 levels are minima w.r.t. E4. Now, with neural networks, your learning algorithms is such that it is guaranteed [yes, mathematical guarantee] to take you to a lesser energy level. The trouble is you don't know which lower state it will put you into. But, oh, lower, it will be!

Source Link
nKarza
nKarza
  • 64
  • 1

@answering to @Media's comment about why there are many local minimas: http://www.chm.bris.ac.uk/pt/allan/elevels.gif

Consider, your randomly initialized weights to be belonging to energy level E4 in above image. Clearly, E1, E2, E3 have lesser energy. So all these 3 levels are minima w.r.t. E4. Now, with neural networks ,your learning algorithms is such that it is guaranteed[yes, mathematical guarantee] to take you to a lesser energy level. The trouble is you don't know which lower state it will put you into. But, oh, lower, it will!