0
$\begingroup$

I am reading the Deep learning book from Ian Goodfellow, etc. My question is about the example in section 17.5 The challenge of mixing between Seperated Modes.

As a simple exampl, consider an energy-based model over two variables a and b, which are both binary with a sign, taking on values −1 and 1. If E(a, b) = −wab. CHAPTER 17. MONTE CARLO METHODSfor some large positive number w, then the model expresses a strong belief that a and b have the same sign. Consider updating b using a Gibbs sampling step with a= 1. The conditional distribution over b is given by P(b= 1| a= 1) =σ(w).If w is large, the sigmoid saturates, and the probability of also assigning b to be 1 is close to 1. Likewise, if a=−1, the probability of assigning b to be −1 is close to 1. According to $P_{model}(a, b)$, both signs of both variables are equally likely. According to $P_{model}(a| b)$, both variables should have the same sign. This means that Gibbs sampling will only very rarely flip the signs of these variables.

I am not sure how P(b= 1| a= 1) = σ(w) gets derived. Please let me know what you think. Any help would be appreciated. Thank you!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.