What is max_trials and executions_per_trial in keras-tuner

Question

I am wondering what is the difference between max_trials & executions_per_trial in kerastuner.tuners.bayesian.BayesianOptimization function. Does execution_per_trial somehow relates to cross valdation? max_trails at first sounds like number of epochs to train but its not the case since there we have a another attribute for that. I didn't get much from the original documentation found here

You should specify the model-building function, the name of the objective to optimize (whether to minimize or maximize is automatically inferred for built-in metrics), the total number of trials (max_trials) to test, and the number of models that should be built and fit for each trial (executions_per_trial).

Answer 1

max_trials represents the number of hyperparameter combinations that will be tested by the tuner, while execution_per_trial is the number of models that should be built and fit for each trial for robustness purposes.

For example, let's imagine you have a shallow network (one hidden layer) with the following parameter search space:

• Number of Hidden units: 16, 32, 48, 64
• Choice of activation functions: ReLu, Sigmoid, TanH

You have a total of 12 combinations (4 times 3).

Then, at this point, when you set up max_trials = 40, you configure the tuner to find 40 random tuples of the # Hidden units and activation functions, e.g. (16, ReLu), (32, Sigmoid), (16, TanH)... up until 40 tuples. For each tuple you will run as many executions as you also set up in execution_per_trial variable, given that depending on how the model runs the optimization process, final results could be very different. For each trial and execution, the tuner will fit the model with as many epochs as you configure in the script.