0
$\begingroup$

When applying a Bayesian inference method such as Gaussian Process Regression (GPR), the assumption of a prior and likelihood function following a normal distribution is inherent. One can use an arbitrary kernel for the covariance function, but nevertheless, a Gaussian is the underlying distribution for the model (as the name implies).

But what is the significance of using a Gaussian distribution besides mathematical convenience? Under what scenarios would using the assumption of Gaussianity be invalid? For example, if a detector measures a time series with error following a Poisson distribution, should GPR be avoided when fitting this data? Namely, I am trying to understand when GPR is applicable and when it breaks down. As far as I can tell, there is no general property or extension of the Central Limit Theorem for why GPR should generalize well when performing functional regression on arbitrary data even with arbitrary kernels. Thus I am trying to gain insight and motivation on when GPR does (and does not) work.

$\endgroup$
0
$\begingroup$

In gaussian process regression, when building the predictor, you specifically use the properties of the gaussian multivariate distribution.

The math part is quite horrible (see here for instance). But the general idea is that a gaussian process regression predictor is a gaussian variable in each point because it is basically a linear combination of other gaussian variables. So the gaussian assumption seems to be necessary.

A possible solution to get rid of the gaussian prior assumption would be to find a transformation of your data, from your custom distribution to a gaussian one. This is apparently named trans-gaussian kriging in the litterature. I haven't had enough time to look at it for now, but here's a possible starting reference studying this matter.

$\endgroup$
  • $\begingroup$ Could you expand upon this: "So I don't think you can easily replace the gaussian assumption of GPRs by any prior distribution"? I believe this is at the heart of my question: why should we make the assumption of a prior following a Gaussian? Similarly, why should we make the assumption of a likelihood function following a Gaussian? When is it valid (and not) to do so? Is it a good model even if the underlying data followed another distribution (e.g. Poisson, Lorentzian)? $\endgroup$ – Mathews24 Oct 14 '18 at 16:24
  • $\begingroup$ I searched further, see my edited answer. $\endgroup$ – Romain Reboulleau Oct 15 '18 at 8:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.