What exactly is convergence rate referring to in machine learning?

Question

My understanding of the term "Convergence Rate" is as follows:

Rate at which maximum/Minimum of a function is reached, so in logistic regression rate at which gradient decent reaches global minimum.

So by convergence rate I am guessing it is measure of:

1. time measured from start of gradient descent until it reaches global maximum.
2. average number of distance our model went downhill(Do not know technical term...) for each iteration.

Can someone verify whether or not one of my guess is True if not explain what it means?

Answer 1

A rate is always a gain per some time/step. A rate can exist even if the maximum is never reached. In supervised learning a loss function is defined, which is expected to have a global maximum, that we try to reach by gradient descent. How much closer we get with each timestep/iteration/epoch/batch is the rate of convergence.