In this example I have a RBM with a visible layer and a hidden layer. The original data is "data", the values of the hidden neurons is "hid", the values of the visible neurons calculated from "hid" is "vis", and then the value for the hidden neurons calculated from "vis"is "hid2".
I calculate the positive_association and the negative_association by:
pos_associations = T.dot(data.T, hid) neg_associations = T.dot(vis.T, hid2)
where T.dot represents the usual matrix calculation. I then update the weights using contrastive divergence (CD)
w = w + (training_rate)*(pos_associations - neg_associations)
like described in this post
I do get a good reconstruction (vis looks like data) and I can see features representing the weights. However, it takes a while and many epochs.
If instead of using CD I simply calculate the quadratic differnce between data and vis
cost = T.sum((data - vis)**2).mean()
and I update the weights using the gradient of cost, I get a faster approximation of the data. So my question is, what is the advantage of using CD?