Questions tagged [rbm]
A restricted Boltzmann machine (RBM) is a stochastic neural network.
31
questions
0
votes
0
answers
6
views
Retricted Boltzmann Machine - difference between wake and dream part - constrastive divergence
Given a restricted boltzmann machine, the update given to the weights is computed as:
<v_ih_j>{data} - <v_ih_j>{model}
In different libraries and implementation, i noticed that there are ...
- 149
0
votes
1
answer
12
views
How interpret or what's the meaning of rbm.up results?
I am studying deep learning and the deepnet R package gives me the following example: (rbm.up function Infer hidden units states by visible units)
...
- 103
2
votes
1
answer
66
views
Build Deep Belief Autoencoder for Dimensionality Reduction
I'm working with a large dataset (about 50K observations x 11K features) and I'd like to reduce the dimensionality. This will eventually be used for multi-class classification, so I'd like to extract ...
- 167
1
vote
1
answer
96
views
Newbie question on restricted boltzmann machine
I’m quite a newbie to RBMs so I’m trying to understand how do you feed real valued data to it given that all the visible and hidden units are binary?
- 113
4
votes
2
answers
130
views
How do I train an RBM on color images?
I am having a hard time understanding the strategy for inputting the color. Most tutorials on RBMs only train grayscale images.
If the image is grayscale, the input units can be binary, and I can ...
- 141
0
votes
1
answer
355
views
What is the difference between reconstruction vs backpropagation?
I was following a tutorial on understanding Restricted Boltzmann Machines (RBMs) and I noticed that they used both the terms reconstruction and ...
- 257
1
vote
3
answers
496
views
What is the difference between symmetric bipartite graphs and a complete bipartite graph?
I am studying Restricted Boltzmann Machines (RBMs), and it is described as a symmetrical bipartite graph. Link
How is this different from a Complete bipartite ...
- 257
1
vote
0
answers
185
views
Model Joint Probability of N Words Appearing Together in a Sentence
Assume that we have a large corpus of texts to train with. Given N words as input, I want to model the joint probability $p(x_1, x_2, ..., x_N)$ of these words appearing together in a sentence. More ...
- 11
1
vote
0
answers
44
views
Training Gaussian Restricted Boltzmann Machines with Noisy Rectified (nrelu or ssu) linear hidden units
I'm not sure how to implement this architecture. I'm following this thesis (pages 17-19) or this paper but I'm not sure how to train it. I want to use this to extract features from raw audio.
I know I ...
- 111
1
vote
1
answer
967
views
How to generate a sample from a generative model like a Restricted Boltzmann Machine?
I am learning about the Boltzmann machine. So far, I have successfully written a code that can learn the coefficients of the energy function of a Restricted Boltzmann Machine. Now, since my model is ...
- 175
3
votes
1
answer
82
views
How can I know the name of the features selected by a Deep Belief Network?
I want to use DBN to reduce the 41 features of nslkdd dataset after transforming nominal data to numeric the number of features increases from 41 to 121 . I used 3 RBMs (121-50-10) now I want to know ...
- 159
2
votes
1
answer
163
views
Isn't computing the "tractable error" in Restricted Boltzmann Machines (RBM) intractable?
Let $v \in \{0,1\}^M$ be the visible layer, $h \in \{0,1\}^N$ be the hidden layer, where $M$ and $N$ are natural numbers. Given the biases $b \in \Re^M$, $c \in \Re^N$ and weights $W \in \Re^{M \times ...
1
vote
1
answer
4k
views
Understanding Contrastive Divergence
I’m trying to understand, and eventually build a Restricted Boltzmann Machine. I understand that the update rule - that is the algorithm used to change the weights - is something called “contrastive ...
- 451
0
votes
1
answer
2k
views
Softmax vs Sigmoid in RBM/Auto Encoder final layer
I'm creating a deep network with 3 hidden layers for classification of the MNIST dataset. The network architecture is something like:
...
- 143
-1
votes
1
answer
318
views
Suitable Autoencoder for Activity Recognition dataset Feature Extraction
I have text data representing sensor outputs.
Dataset:
...
- 99
1
vote
1
answer
311
views
How hidden units are conditionally independent in restricted Boltzmann machines
Why explaining away concept is not applicable in restricted Boltzmann machines? Their hidden units form a V structure from which probabilistic influence can flow given the observed visible variable. ...
- 11
2
votes
1
answer
921
views
Why training a Restricted Boltzmann Machine corresponds to having a good reconstruction of training data?
Many tutorials suggest that after training a RBM, one can have a good reconstruction of training data just like an autoencoder.
An example tutorial.
But the training process of RBM is essentially to ...
- 23
1
vote
0
answers
509
views
Understanding a simple example of Restricted Boltzmann Machine (RBM)
I am trying to get an intuitive idea of RBMs out of curiosity, and using a simple example on youtube based on preferences for different sports, which denote profiles roughly corresponding to ...
- 111
2
votes
1
answer
1k
views
How hidden layer is made binary in Restricted Boltzmann Machine (RBM)?
In RBM, in the positive phase for updating the hidden layer(which should also be binary), [Acually consider a node of h1 ∈ H(hidden layer vector)] to make h1 a binary number we compute the probability ...
- 337
1
vote
1
answer
435
views
# of iterations in Restricted Boltzmann Machine (RBM)
I have a training set, I provide it (consider a data from training set) to the visible layer. Then the normal process is followed, i.e. Positive Phase-> Negative Phase-> Reconstruction of weights, ...
- 337
16
votes
3
answers
2k
views
Intuition Behind Restricted Boltzmann Machine (RBM)
I went through Geoff Hinton's Neural Networks course on Coursera and also through introduction to restricted boltzmann machines, still I didn't understand the intuition behind RBMs.
Why do we need ...
- 337
3
votes
1
answer
2k
views
How is dimensionality reduction achieved in Deep Belief Networks with Restricted Boltzmann Machines?
In neural networks and old classification methods, we usually construct an objective function to achieve dimensionality reduction. But Deep Belief Networks (DBN) with Restricted Boltzmann Machines (...
- 33
14
votes
3
answers
10k
views
How to use RBM for classification?
At the moment I'm playing with Restricted Boltzmann Machines and since I'm at it I would like try to classify handwritten digits with it.
The model I created is now a quite fancy generative model but ...
- 243
2
votes
1
answer
727
views
Pre-train using sigmoid and train using ReLU?
Using RBMs to pre-train a deep net as in this example RBM, the activation function is sigmoid and makes the math much easier.
What are the implications after the initial weights are learned using ...
- 121
4
votes
3
answers
2k
views
Training Restricted Boltzmann Machines (RBMs) using gradient descent
Hey I am a little new to the whole RBM entropy/energy training thing, having some trouble understanding and trying to figure whether it is worth the effort needed to understand.
Can't RBMs quite ...
- 41
1
vote
1
answer
77
views
Hidden neuron representation of weights
In an RBM, if we represent the weights learned by the hidden units, they show that the neural net is learning basic shapes. For example, for the mnist dataset, they learn features of the numbers they ...
- 1,973
3
votes
1
answer
142
views
Training the parameters of a Restricted Boltzman machine
Why are the parameters of a Restricted Boltzmann machine trained for a fixed number of iterations (epochs) in many papers instead of choosing the ones corresponding to a stationary point of the ...
- 205
7
votes
2
answers
482
views
How are non-restricted Boltzmann machines trained?
Restricted Boltzmann machines are stochastic neural networks. The neurons form a complete bipartite graph of visible units and hidden units. The "restricted" is exactly the bipartite property: There ...
- 18.8k
5
votes
2
answers
730
views
What are the advantages of contrastive divergence vs the gradient of the quadratic difference between the original data and the reconstructed data?
In this example I have a RBM with a visible layer and a hidden layer. The original data is "data", the values of the hidden neurons is "hid", the values of the visible neurons calculated from "hid" is ...
- 1,973
2
votes
0
answers
234
views
Contrastive divergence in RBM
I have the following code, where x in the input data, w is the weight matrix, bv and bh are the biases for the visible and hidden units.
...
- 1,973
8
votes
1
answer
970
views
Why a restricted Boltzman machine (RBM) tends to learn very similar weights?
These are 4 different weight matrices that I got after training a restricted Boltzman machine (RBM) with ~4k visible units and only 96 hidden units/weight vectors. As you can see, weights are ...
- 2,801