0
$\begingroup$

I would like to find optimal combination of parameters for the algorithm affecting the disk space used by some storage. Therefore, several algorithm parameters (x1, x2, x3, where 0 < x1 < 1, 10 < x2 < 100, 0.1 < x3 < 0.5) are used as an input for the model, and the disk space occupied by storage S(x1, x2, x3) is the cost function I'd like to minimize.

The problem is that every function call S(x1, x2, x3) is very expensive, and it may take minutes or even hours to end up, so it's almost infeasible to prepare the training set. Instead, I would like ML library to suggest, which combination of (x1, x2, x3) values to use to estimate S on every next optimization step.

I believe this task is quite common for the major ML frameworks, but I wasn't able to find anything suitable yet. So what is the name for this class of problems, and which ML framework provide the routines to solve it?

Improve this question
$\endgroup$
2
  • $\begingroup$ If I understood correctly, you would not like to estimate your cost function during the optimization because it is very computationally expensive, but rather use an approximation, based on some previous/historical data and a model built upon this data? If so, do you already have any such data, mapping your parameters with the output? Or are you asking us how to obtain such data in the first place? $\endgroup$
    – user2974951
    Commented Dec 13, 2021 at 13:24
  • $\begingroup$ @user2974951, no, of course, it's necessary to estimate cost function during the optimization. The difference is that in this task it isn't possible to use the common approach, when you first generate large dataset for S using various (x1, x2, x3) tuples, and then use trivial gradient descent to find the minimum. The first step would take too long here, so I just want gradient descent to provide (x1, x2, x3) combination to calculate S on every step according to current gradient values $\endgroup$
    – Vitaly Isaev
    Commented Dec 13, 2021 at 13:57

1 Answer 1

Reset to default
1
$\begingroup$

What you have is a standard optimization problem, which has got nothing to do with ML. I don’t know what modelling language you use, but Scipy in Python offers many nonlinear optimization algorithms - global or local.

If I were you, for this problem I would stay away as much as possible from gradient descent: if the evaluation of your cost function does not also provide gradient information, then you have to calculate it numerically. Which involves many more functions evaluations. Whether your cost function provides gradient outputs or not, gradient descent is just about the worst optimization algorithm you can think of in the realm of nonlinear solvers.

Global solvers will attempt to give you a “global optimum” - but they are much more expensive to run and you will never know whether the minimum/maximum you found is the actual “global” one. Local algorithms are faster to converge but they only ever provide a locally optimum minimum/maximum.

https://docs.scipy.org/doc/scipy/reference/optimize.html

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.