I have a problem of a task using the formula of the Gradient Descent:

Gradient Descent

Perform two steps of the gradient descent towards a local minimum for the function given below, using a step size of 0.1 and an initial value of [1, 1]


I only get as result of the derivative 0.4x1, because x2 does not exist. Is this correct or should the result for the derivative be (0.4, 0)?

Note: Sorry, if my equation of the derivative is bad. I'm not a mathematician. Please, correct me, if my equation is absolutely wrong.

  • 1
    $\begingroup$ Yes, it is correct. You have only one variable in the function equation. The initial point should be [1, 200.2] $\endgroup$ – 10xAI Jul 19 '20 at 11:51
  • $\begingroup$ How to calculate the second value 200.2? $\endgroup$ – Ramón Wilhelm Jul 19 '20 at 12:46
  • $\begingroup$ 200.2 is just the f(x) for x=1. The point which will lie on a function will be (x, y) so [1, 200.2] will lie on the function. You can get the Gradient and then the next point and then Repeat. The gradient will have as many components as in the function(some may be zero) but it can't have more components than the function itself. $\endgroup$ – 10xAI Jul 19 '20 at 13:14
  • $\begingroup$ This make sense. But I though you first have to compute the derivative, before you change the x to 1. $\endgroup$ – Ramón Wilhelm Jul 19 '20 at 13:20
  • 1
    $\begingroup$ Yes, Gradient will be 0.4x only at point (1, 202.2). In short so that you don't get confuse, go ahead calculate the gradient and move towards the minima. $\endgroup$ – 10xAI Jul 19 '20 at 13:47

Gradient, g(x) = 0.4*x

At [x1 = 1], Gradient,
g(1) = 0.4

x2 = x1- step*gradient
=>x2 = 1 - 0.1*0.4
=>x2 = 0.96

At [x2 = 0.96], Gradient,
g(0.96) = 0.4*0.96 = 0.384
=> x3 = 0.96 - 0.1*0.384 = 0.9216

Continue following the same steps and will reach near the minima.

  • $\begingroup$ Thank you very much. I'm not confused anymore! :) $\endgroup$ – Ramón Wilhelm Jul 19 '20 at 15:19
  • 1
    $\begingroup$ Just to add, had it been 2 var x1, x2 for f(x)(Since it was your initial query). We would have calculated 2 Gradients(i.e. 2 partial derivatives), 2 separate subtraction, and would have got two new X1, X2 in the same manner. The process will remain same. $\endgroup$ – 10xAI Jul 19 '20 at 15:24
  • $\begingroup$ All right. I keep this in mind. Normally, with the vars X1 and X2 it would not be so difficult for me. I have calculated this exercise again by myself and I fully understand it now thanks to you! :) $\endgroup$ – Ramón Wilhelm Jul 19 '20 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.