# What exactly is convergence rate referring to in machine learning?

My understanding of the term "Convergence Rate" is as follows:

Rate at which maximum/Minimum of a function is reached, so in logistic regression rate at which gradient decent reaches global minimum.

So by convergence rate I am guessing it is measure of:

1. time measured from start of gradient descent until it reaches global maximum.
2. average number of distance our model went downhill(Do not know technical term...) for each iteration.

Can someone verify whether or not one of my guess is True if not explain what it means?

• This blog might help. May 11 at 7:37

## 1 Answer

A rate is always a gain per some time/step. A rate can exist even if the maximum is never reached. In supervised learning a loss function is defined, which is expected to have a global maximum, that we try to reach by gradient descent. How much closer we get with each timestep/iteration/epoch/batch is the rate of convergence.

• Thanks! I could not find any implementation of extracting learning rate in sklearn, do you know how? It is a question I've asked on stackOverflow -> stackoverflow.com/questions/67481939/… May 11 at 7:44
• I don't think there is any. The rate is more of a concept that is oftentimes just eyeballed by looking at the loss function over time, e.g. in a plot. May 11 at 7:45
• doesn't using gradient descent allow us to reach global minimum? for example in logistic regression our goal is to minimize loss function. May 11 at 7:50