2
$\begingroup$

I read the likelihood is defined in logistic regression as the probability $$ L(w) = P(y|x, w) = \prod P(y^i| x^i,w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ and the log of the last equation is:

$$ log(L(w)) = \sum y^i log((\sigma(z^i)) + (1-y^i)log(1-\sigma(z^i)) $$

I understand (independent probabilities) $$ L(w) = P(y|x, w) = \prod P(y^i| x^i,w) $$ and I understand that from $$ L(w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ the log is (basic log properties) $$ log(L(w)) = \sum y^i log((\sigma(z^i)) + (1-y^i)log(1-\sigma(z^i)) $$ However, how do I get $$ \prod P(y^i| x^i,w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ This basically means that $$ P(y| x,w) = (\sigma(z))^{y}(1-\sigma(z))^{(1-y)} $$ and I just don't see that.

$\endgroup$

1 Answer 1

1
$\begingroup$

By the definition of the logistic regression model $\mathrm P(y = 1 | x,w) = \sigma\left(\left<w, x\right>\right)$ thus $\mathrm P(y = 0 | x,w) = 1- \sigma\left(\left<w, x\right>\right)$

You should be able to verify (by setting y=0/y=1) that this is equivalent to

$\mathrm P(y | x,w) = \sigma\left(\left<w, x\right>\right)^y \left[ 1- \sigma \left( \left<w, x \right> \right) \right]^{1-y}$

I think you understand the rest?

$\endgroup$
1
  • $\begingroup$ Yes, I see, it is just a way to compact everything into a single expression. $\endgroup$
    – user
    May 3, 2016 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.