In RBM, in the positive phase for updating the hidden layer(which should also be binary), [Acually consider a node of h1 ∈ H(hidden layer vector)] to make h1 a binary number we compute the probability of turning on a hidden unit by operating activation function over total input (after the activation function operation, we would be getting values in the range between 0 and 1, since activation function I am using - sigmoid). My doubt is that how do we make it binary by leveraging the probability computed. I don't think if P>=0.5, make it 1 else 0 is a proper method to work on.
By few literature reviews, I found this document (by Hinton), in section 3.1: he has stated "the hidden unit turns on if this probability is greater than a random number uniformly distributed between 0 and 1". What does this actually mean? And also in this link, they say "Then the jth unit is on if upon choosing s uniformly distributed random number between 0 and 1 we find that its value is less than sig[j]. Otherwise it is off." I actually didn't get this. Whether the random number generated is same for all h ∈ H ? Another query is, what about the random in next sampling iteration?
I saw this video. Just watch the video from that point as per the link. How do you get that sampled number? Whether we have to just run rand() in Matlab and obtain it? Should it would be different for each h(i) (oh nooo! I don't think the machine will learn properly)? Whether the random number should be different for each iteration or the same random number can be used for all iteration to compare?