This is my first question on this forum. I just got started with Bayesian statistics. While I do understand the motivations behind Bayesian methods, I am a little unclear on what the predictions even mean. Consider a standard regression problem of predicting the price of a house given its area in square feet. Assume the optimal parameters (slope and intercept) of the model have been found. The price of any new house (x_new) is just the number - {intercept + slope * x_new}.

In Bayesian linear regression, we work with the so-called posterior predictive distribution (abbreviated PPD). But what is the PPD anyway?

1) Is it a probability density function (pdf) with some parameters like mean and (co)variance? If so, how do I obtain a single value of the house price from this density function? Should I just take the mean of this distribution or are there sophisticated techniques available?

2) Is it a real number, given by the equation - intercept + slope * x_new? If so, a. Are intercept and slope sampled from the posterior distribution of slope and intercept? b. Or are the posterior mean values of slope and intercept used for computing the price of the new house?


Let me help you get started -

  1. you don't get a single value here like you get in linear regression,rather posterior distribution of the parameters. yes, The output here is generated from a normal (Gaussian) Distribution characterized by a mean and variance , nothing special I am afraid. But depending on the mean and variance you can determine which price is more likely to be.

  2. yes its not a real valued number like price here but the probability ,output is the likelihood multiplied by the prior probability of the data.

Here is a very simple explanation, I couldn't put it in better words-


And here is the more mathematical background with this chapter from stanford-


and you are not alone as these Author's agree-

"Gaussian process regression models, though perhaps somewhat tricky to understand conceptually, nonetheless lead to simple and straightforward linear algebra implementations"

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.