I am using Gaussian Process Regressor to fit data for a Bayesian Optimiser. This is a relevant part of my Python code.

from sklearn.gaussian_process import GaussianProcessRegressor
from sklearn.gaussian_process.kernels import (RBF, Matern, RationalQuadratic,
                                          ExpSineSquared, DotProduct,
kernel = Matern(length_scale=lenSc,nu=2.5)
gp = GaussianProcessRegressor(kernel=kernel)
yi=map(lambda x: f1(x), Xi)
gp.fit(Xi, yi)
   #Some code to get the new values- cc and nv
   gp.fit(Xi, yi)

Each iteration of the Bayesian Optimiser, I add a new element to $Xi$ and $yi$ and fit $gp$, the Gaussian Process, again. As the size of $Xi$ and $yi$ reach over 200, the time taken to fit the data becomes noticeable, this decreases the overall efficiency of my Bayesian Optimiser.

The Bayesian Optimiser has to make around 1000 iterations, each iteration needs to fit the data all over again. So, I was wondering if there's a way to incrementally fit the data? If that's not already being done by sklearn's GaussianProcessRegressor.

Thank you!

  • 1
    $\begingroup$ stats.stackexchange.com/questions/153938/… $\endgroup$
    – Emre
    Commented Aug 19, 2016 at 17:46
  • $\begingroup$ Thank you! I was hoping for something more like an existing library or something. Unfortunately, I don't think I will have the time to implement these papers. $\endgroup$ Commented Aug 21, 2016 at 13:52
  • $\begingroup$ Did you find a suitable library? $\endgroup$
    – user77005
    Commented Nov 21, 2018 at 14:38

1 Answer 1


I think with GPR, the computational cost increases on an order of N^3, so the amount of time with more data increases drastically.

I don't think the current version of sklearn's GPR implements the kind of incremental fit you are expecting. With larger data and iterations, I suggest implementing your own version on a GPU or somethings since GPR implementation is not that difficult.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.