2
$\begingroup$

I read the likelihood is defined in logistic regression as the probability $$ L(w) = P(y|x, w) = \prod P(y^i| x^i,w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ and the log of the last equation is:

$$ log(L(w)) = \sum y^i log((\sigma(z^i)) + (1-y^i)log(1-\sigma(z^i)) $$

I understand (independent probabilities) $$ L(w) = P(y|x, w) = \prod P(y^i| x^i,w) $$ and I understand that from $$ L(w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ the log is (basic log properties) $$ log(L(w)) = \sum y^i log((\sigma(z^i)) + (1-y^i)log(1-\sigma(z^i)) $$ However, how do I get $$ \prod P(y^i| x^i,w) = \prod (\sigma(z^i))^{y^i}(1-\sigma(z^i))^{(1-y^i)} $$ This basically means that $$ P(y| x,w) = (\sigma(z))^{y}(1-\sigma(z))^{(1-y)} $$ and I just don't see that.

$\endgroup$
1
$\begingroup$

By the definition of the logistic regression model $\mathrm P(y = 1 | x,w) = \sigma\left(\left<w, x\right>\right)$ thus $\mathrm P(y = 0 | x,w) = 1- \sigma\left(\left<w, x\right>\right)$

You should be able to verify (by setting y=0/y=1) that this is equivalent to

$\mathrm P(y | x,w) = \sigma\left(\left<w, x\right>\right)^y \left[ 1- \sigma \left( \left<w, x \right> \right) \right]^{1-y}$

I think you understand the rest?

$\endgroup$
1
  • $\begingroup$ Yes, I see, it is just a way to compact everything into a single expression. $\endgroup$ – user May 3 '16 at 19:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.