Could someone explain in simple words
- what are gaussian processes
- how does bayesian optimization work
- and their combination?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
Gaussian processes are a way of interpolating between data using a kernel fit, but with a covariance matrix. Think of it as kernel regression, but with confidence intervals that can be drawn at every point.
This diagram from this medium article is explains how a combination of a Gaussian process and an acquisition function works in Bayesian optimization:
An optimization problem is one that has an objective, for example, you might want to find a global minimum. Given the predictions and the confidence interval of a Gaussian process, Bayesian optimization uses an acquisition function to choose observations that would advance that objective. The acquisition function is a combination of that objective, and a balance between exploration and exploitation.
The key concepts you should grasp are: