All supervised learning techniques have some kind of an inductive bias. What is the inductive bias in Gaussian process models ?


The inductive bias of a Gaussian process (GP) is encoded in the covariance kernel. A GP is a distribution over functions — when we choose a kernel, we are specifying characteristics that we expect the solution function to have, e.g., smooth, linear, periodic, etc.

For example, a common covariance kernel is the squared exponential:

$$k_{SE}(x, x') = \exp \left( - \frac{||x - x'||_2^2}{2l^2} \right)$$

where $l$ is the characteristic length-scale. This kernel specifies that an unobserved point will be similar to each observed point as an exponentially decaying function of the Euclidean distance between the two points.

More detailed explanations can be found in section 2.4 of Andrew Gordon Wilson's thesis or in the book Gaussian Processes for Machine Learning.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.