What is the num_initial_points argument for Bayesian Optimization with Keras Tuner?

I've implemented the following code to run Keras-Tuner with Bayesian Optimization:

def model_builder(hp):

NormLayer = Normalization()

model = Sequential()

for i in range(hp.Int('conv_layers',2,4)):

for i in range(hp.Int('dense_layers',1,2)):

earlystop = EarlyStopping(monitor='val_loss',patience=8,restore_best_weights=True)

model.compile(loss='categorical_crossentropy',optimizer=opt,metrics=['accuracy'])

return model

tuner = BayesianOptimization(model_builder,objective='val_loss',num_initial_points=??,max_trials=tuner_trials,directory='BayesianOptimization/',project_name='BayesianOptimization')


What do the num_initial_points argument does exactly and what should I set it to in my case?

Reading the documentation I see the description

The number of randomly generated samples as initial training data for Bayesian optimization

but not being an expert I don't exactly get what it means and how it will impact the optimization process.

Setting a high number of random points gives you guaranteed "exploration" points; indeed, in the documentation for the package bayesian-optimization, we find:
init_points: How many steps of random exploration you want to perform. Random exploration can help by diversifying the exploration space.
(The default is 5 in that package, and 3 times the dimension in keras-tuner.) That said, you can also make the algorithm focus more or less on exploration/exploitation directly, using the beta parameter (kappa in bayesian-optimization, see this example notebook).
• @WVJoe That's my understanding, yes. But now I see that I've been referencing bayesian-optimization, not the question's keras-tuner version. I'll edit my answer a bit. – Ben Reiniger Feb 17 at 3:45