Optimising Kernel parameters using training data in GaussianProcessRegressor of Scikit-learn

Question

I want to optimize the Kernel parameters or hyper-parameters using my training data in GaussianProcessRegressor of Scikit-learn.Following is my query:

My training datasets are:

X: 2-D Cartesian coordinate as input data

y: radio signal strength (RSS) at the 2-D coordinates points as observed output

What I've done so far:

I've installed python and Scikit-learn software. I've successfully tested the sample codes. I'm able to predict RSS at test points using training data. I use squared exponential Kernel.

What I want to do:

I want to train the Kernel parameters (hyper-parameter) with different optimizing algorithms like gradient descent, swarm intelligence, and trust-region-reflective algorithms.

What I learned and What help I am asking for:

I've learned that, in the GaussianProcessRegressor class of scikit, the optimizer is an argument where I can use my own optimizing algorithm. Since it is callable, I need to write my own function/method for it. Can I use any inbuilt library (library of optimization algorithm) in GaussianProcessRegressor class? Are there such libraries available for python? Could anybody provide any sample code for using kernel parameter optimization algorithm in GaussianProcessRegressor? I've learned that we use the training datasets for optimizing the hyper-parameters. Could anybody provide any insight about relating the training datasets with the optimization algorithm, please?

Thank you!

Answer 1

Given scikit-learn's API, you create a separate instance for each optimizer and compare the results to see which optimizer makes better predictions. It would looks something like:

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import RBF

gp1 = GaussianProcessRegressor(kernel=RBF, optimizer=optimizer1)
gp1.fit(X, y)
y_pred1 = gp1.predict(x)

gp2 = GaussianProcessRegressor(kernel=RBF, optimizer=optimizer2)
gp2.fit(X, y)
y_pred2 = gp2.predict(x)

This blog post gives a general overview of fitting Gaussian Processes, including scikit-learn's GaussianProcessRegressor.