1
$\begingroup$

A posterior $p(\theta\vert x) = \frac{p(x \vert \theta)p(\theta)}{p(x)} $

Many materials say that since the $p(x)$ is a constant, the $p(x)$ can be ignored. Thus, $p(\theta\vert x) \propto p(x \vert \theta)p(\theta)$

My question is why $p(x)$ is a constant and ignored. Is this because even though we don't know the distribution x, there is a corresponding true distribution for $x$? So, $p(x) $ is a constant (we don't know but already determined) and thus, can be ignored?

$\endgroup$
  • $\begingroup$ Oh, I made the mistake $p(x)$ is not a prior. Prior is $p(\theta)$. I edited my question! $\endgroup$ – shashack Mar 1 at 5:33
1
$\begingroup$

You are right, $P(x)$ is the underlying distribution of data, and it can be assumed constant, simply because it is independent of our modeling practice. However $P(x)$ may change gradually over time, this phenomenon is called "distribution shift". For example, the distribution of height in a country may change over years.

Note that "prior" is a reserved word for $P(\theta)$ which is the distribution of model parameters in Bayesian modeling. In non-Bayesian modeling there is no notion of "prior". $P(x|\theta)$ is called likelihood and $P(x)=\sum_{\theta}P(x|\theta)P(\theta)$ is called marginalized likelihood (likelihood marginalized over model parameters).

$\endgroup$
1
$\begingroup$

The maximum a posteriori definition usually goes like this:

$$ \arg\max_{\theta} p(\theta|x) = \arg\max_{\theta} \frac{p(x|\theta)\cdot p(\theta)}{p(x)} $$

and given that $p(x)$ is independent of $\theta$, it's not needed for finding the $\arg\max_{\theta}$, then you have

$$ \arg\max_{\theta} p(\theta|x) = \arg\max_{\theta} \frac{p(x|\theta)\cdot p(\theta)}{p(x)} = \arg\max_{\theta} p(x|\theta)\cdot p(\theta)$$

$\endgroup$
1
$\begingroup$

Yes. Presumably whatever process 'generates' data produces data that follows some distribution. $p(x)$, however, is just a number, and whatever it is, it is a constant w.r.t. $\theta$.

I wouldn't call $p(x)$ a prior, as it implies there's some posterior probability of the data we compute. We don't; nothing about this process updates belief about the probability of the. "Prior" refers to $p(\theta)$.

$p(x)$ can't be ignored if you really want the value of $p(\theta|x)$. But usually you compute that from the posterior distribution that you will have a closed-form formula for.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.