0
$\begingroup$

I saw multiple articles describes GD or SGD with the following diagram: enter image description here

I didn't saw any explanation about the ellipses.

  1. What does each ellipse describe ?
  2. What is the meaning if the ellipse has large or small diameter ?
  3. Does the wights (which are the ellipse in the image) must has the ellipse shape ? (Are there any other shapes you can show as an example) ?
Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$
  1. These ellipse represent contour of the Gradient Descent The ellipse line have same altitude/height in the Gradient.

lets understant it with help of mountain analogy. The mountain is your objective function which you want to maximize or minimize. You want to reach the top of mountain, how is the top represented in countour plot - its the center of ellipses. enter image description here

  1. The Diameter tells us about the shape of the function and the distance between two ellipse tells us how fast or slow the slope decreases. For example lets take the slope between two successive ellipse be 5 meters in height so if the ellipses are closer together we can conclude that altitude is falling very sharply and vice-versa.

  2. Other shape is also possible. It depends upon the objective function which you are trying to minimize/maximize. enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.