I want to optimize the Kernel parameters or hyper-parameters using my training data in GaussianProcessRegressor of Scikit-learn.Following is my query:

My training datasets are:

X: 2-D Cartesian coordinate as input data

y: radio signal strength (RSS) at the 2-D coordinates points as observed output

What I've done so far:

I've installed python and Scikit-learn software. I've successfully tested the sample codes. I'm able to predict RSS at test points using training data. I use squared exponential Kernel.

What I want to do:

I want to train the Kernel parameters (hyper-parameter) with different optimizing algorithms like gradient descent, swarm intelligence, and trust-region-reflective algorithms.

What I learned and What help I am asking for:

I've learned that, in the GaussianProcessRegressor class of scikit, the optimizer is an argument where I can use my own optimizing algorithm. Since it is callable, I need to write my own function/method for it. Can I use any inbuilt library (library of optimization algorithm) in GaussianProcessRegressor class? Are there such libraries available for python? Could anybody provide any sample code for using kernel parameter optimization algorithm in GaussianProcessRegressor? I've learned that we use the training datasets for optimizing the hyper-parameters. Could anybody provide any insight about relating the training datasets with the optimization algorithm, please?

Thank you!


1 Answer 1


Given scikit-learn's API, you create a separate instance for each optimizer and compare the results to see which optimizer makes better predictions. It would looks something like:

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF

gp1 = GaussianProcessRegressor(kernel=RBF, optimizer=optimizer1)
gp1.fit(X, y)
y_pred1 = gp1.predict(x)

gp2 = GaussianProcessRegressor(kernel=RBF, optimizer=optimizer2)
gp2.fit(X, y)
y_pred2 = gp2.predict(x)

This blog post gives a general overview of fitting Gaussian Processes, including scikit-learn's GaussianProcessRegressor.

  • $\begingroup$ I read the blog. It explains the basic Gaussian process regression using scikit learn. As mentioned in the blog and given in scikit -learn documentation, L-BFGS-B algorithm (optimizer='fmin_l_bfgs_b') is used to optimize the hyperparameter. Are there any other algorithms that we can use like the L-BFGS-B algorithm in sklearn? To be precise, I want to use trust-region-reflective algorithm. Is it available in sklearn? $\endgroup$
    – santobedi
    May 8, 2018 at 7:45
  • 1
    $\begingroup$ sklearn has a limited selection of optimizers. scipy has many more docs.scipy.org/doc/scipy-0.13.0/reference/generated/… $\endgroup$ May 8, 2018 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.