I am reading the Deep learning book from Ian Goodfellow, etc. My question is about the example in section 17.5 The challenge of mixing between Seperated Modes.

As a simple exampl, consider an energy-based model over two variables a and b, which are both binary with a sign, taking on values −1 and 1. If E(a, b) = −wab. CHAPTER 17. MONTE CARLO METHODSfor some large positive number w, then the model expresses a strong belief that a and b have the same sign. Consider updating b using a Gibbs sampling step with a= 1. The conditional distribution over b is given by P(b= 1| a= 1) =σ(w).If w is large, the sigmoid saturates, and the probability of also assigning b to be 1 is close to 1. Likewise, if a=−1, the probability of assigning b to be −1 is close to 1. According to $P_{model}(a, b)$, both signs of both variables are equally likely. According to $P_{model}(a| b)$, both variables should have the same sign. This means that Gibbs sampling will only very rarely flip the signs of these variables.

I am not sure how P(b= 1| a= 1) = σ(w) gets derived. Please let me know what you think. Any help would be appreciated. Thank you!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.