# Is there a rule of thumb for a sufficient number of trials for hyperparameter search

I am implementing a quite complicated Bayesian hyperparameter search in hyperopt library on a CNN.

Is there a rule of thumb for a "sufficient" number of trials? Perhaps based on the number of hyperparameters that constitute the search space?

• Nice question; about your your comment, how do you evaluate the possible number of params of the search space in case some of those params are continuous variables – German C M Jan 13 at 14:12
• Nice question too! This deserves a question by itself, I suggest you to write one because it'd be very helpful for the whole community. – Leevo Jan 13 at 15:29

So, to conclude...no, I don't actually have a rule of thumb. 60 is probably a decent bet, given the tradeoff of the last two paragraphs; I'd go up to 100 if the training isn't too expensive. Also consider whether your package allows you to continue a Bayesian search: you could analyze the results so far (mostly, see if the last few iterations have clustered around some point, and how widely the objective function varies) to see if you want to proceed. Finally, note that scikit-optimize sets a default of 50, but there doesn't seem to be much behind that.
• Yeah, hyperopt allows to save past trials in a Trials() object, that you can then unpickle and restore for the next search session. Highly suggested btw – Leevo Jan 13 at 15:32