0
$\begingroup$

My understanding of the term "Convergence Rate" is as follows:

Rate at which maximum/Minimum of a function is reached, so in logistic regression rate at which gradient decent reaches global minimum.

So by convergence rate I am guessing it is measure of:

  1. time measured from start of gradient descent until it reaches global maximum.
  2. average number of distance our model went downhill(Do not know technical term...) for each iteration.

Can someone verify whether or not one of my guess is True if not explain what it means?

$\endgroup$
1
1
$\begingroup$

A rate is always a gain per some time/step. A rate can exist even if the maximum is never reached. In supervised learning a loss function is defined, which is expected to have a global maximum, that we try to reach by gradient descent. How much closer we get with each timestep/iteration/epoch/batch is the rate of convergence.

$\endgroup$
3
  • $\begingroup$ Thanks! I could not find any implementation of extracting learning rate in sklearn, do you know how? It is a question I've asked on stackOverflow -> stackoverflow.com/questions/67481939/… $\endgroup$ – haneulkim May 11 at 7:44
  • 1
    $\begingroup$ I don't think there is any. The rate is more of a concept that is oftentimes just eyeballed by looking at the loss function over time, e.g. in a plot. $\endgroup$ – N. Kiefer May 11 at 7:45
  • $\begingroup$ doesn't using gradient descent allow us to reach global minimum? for example in logistic regression our goal is to minimize loss function. $\endgroup$ – haneulkim May 11 at 7:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.