1
$\begingroup$

I believe term ‘learning rate schedule’ is a certain solution for tuning the learning rate. But at the same time, every parameter evaluating the parameter itself can be called a hyperparameter. So can the learning rate schedule be called a hyperparameter?

$\endgroup$
1
$\begingroup$

Hyper-parameters are essentially parameters that you have to tune manually and aren't learned.

The schedule can be considered a hyper-parameter, as well as all the scheduler's parameters (patience, decay, etc.) and even the decision to use or not a scheduler can be considered a hyperparameter.

| improve this answer | |
$\endgroup$
0
$\begingroup$

Learning rate scheduling can be achieved through multiple ways e.g. 1 cycle, power, performance.

Selecting the approach is first Hyperparameter.

Then, selecting the appropriate parameter value for the approach will be another set of Hyperparameters. e.g. eta0, step size etc.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.