Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [rbm]

A restricted Boltzmann machine (RBM) is a stochastic neural network.

0
votes
0answers
8 views

Shuffle dataset of sequences for Restricted Boltzmann Machines

Suppose I want to train an RBM (or even a DBN architecture) and then fine-tune the parameter training a Feedforward NN. In my case, the dataset is composed of time series, so in principle there is a ...
1
vote
1answer
25 views

Newbie question on restricted boltzmann machine

I’m quite a newbie to RBMs so I’m trying to understand how do you feed real valued data to it given that all the visible and hidden units are binary?
0
votes
0answers
13 views

What is a “mean-field reconstruction” in contrastive divergence learning?

I am reading the following paper: An Efficient Learning Procedure for Deep Boltzmann Machines by Salakhutdinov and Hinton. In Algorithm 3, page 1987, there is a term which is "contrastive divergence ...
2
votes
0answers
22 views

How do I train an RBM on color images?

I am having a hard time understanding the strategy for inputting the color. Most tutorials on RBMs only train grayscale images. If the image is grayscale, the input units can be binary, and I can ...
0
votes
1answer
36 views

What is the difference between reconstruction vs backpropagation?

I was following a tutorial on understanding Restricted Boltzmann Machines (RBMs) and I noticed that they used both the terms reconstruction and ...
1
vote
2answers
31 views

What is the difference between symmetric bipartite graphs and a complete bipartite graph?

I am studying Restricted Boltzmann Machines (RBMs), and it is described as a symmetrical bipartite graph. Link How is this different from a Complete bipartite ...
1
vote
0answers
29 views

Model Joint Probability of N Words Appearing Together in a Sentence

Assume that we have a large corpus of texts to train with. Given N words as input, I want to model the joint probability $p(x_1, x_2, ..., x_N)$ of these words appearing together in a sentence. More ...
1
vote
0answers
23 views

Training Gaussian Restricted Boltzmann Machines with Noisy Rectified (nrelu or ssu) linear hidden units

I'm not sure how to implement this architecture. I'm following this thesis (pages 17-19) or this paper but I'm not sure how to train it. I want to use this to extract features from raw audio. I know ...
0
votes
1answer
151 views

How to generate a sample from a generative model like a Restricted Boltzmann Machine?

I am learning about the Boltzmann machine. So far, I have successfully written a code that can learn the coefficients of the energy function of a Restricted Boltzmann Machine. Now, since my model is ...
1
vote
1answer
41 views

How can I know the name of the features selected by a Deep Belief Network?

I want to use DBN to reduce the 41 features of nslkdd dataset after transforming nominal data to numeric the number of features increases from 41 to 121 . I used 3 RBMs (121-50-10) now I want to know ...
2
votes
1answer
65 views

Isn't computing the “tractable error” in Restricted Boltzmann Machines (RBM) intractable?

Let $v \in \{0,1\}^M$ be the visible layer, $h \in \{0,1\}^N$ be the hidden layer, where $M$ and $N$ are natural numbers. Given the biases $b \in \Re^M$, $c \in \Re^N$ and weights $W \in \Re^{M \times ...
0
votes
1answer
1k views

Understanding Contrastive Divergence

I’m trying to understand, and eventually build a Restricted Boltzmann Machine. I understand that the update rule - that is the algorithm used to change the weights - is something called “contrastive ...
0
votes
1answer
502 views

Softmax vs Sigmoid in RBM/Auto Encoder final layer

I'm creating a deep network with 3 hidden layers for classification of the MNIST dataset. The network architecture is something like: ...
0
votes
1answer
107 views
1
vote
1answer
152 views

How hidden units are conditionally independent in restricted Boltzmann machines

Why explaining away concept is not applicable in restricted Boltzmann machines? Their hidden units form a V structure from which probabilistic influence can flow given the observed visible variable. ...
2
votes
1answer
437 views

Why training a Restricted Boltzmann Machine corresponds to having a good reconstruction of training data?

Many tutorials suggest that after training a RBM, one can have a good reconstruction of training data just like an autoencoder. An example tutorial: https://deeplearning4j.org/...
1
vote
0answers
360 views

Understanding a simple example of Restricted Boltzmann Machine (RBM)

I am trying to get an intuitive idea of RBMs out of curiosity, and using a simple example on youtube based on preferences for different sports, which denote profiles roughly corresponding to ...
2
votes
1answer
808 views

How hidden layer is made binary in Restricted Boltzmann Machine (RBM)?

In RBM, in the positive phase for updating the hidden layer(which should also be binary), [Acually consider a node of h1 ∈ H(hidden layer vector)] to make h1 a binary number we compute the probability ...
1
vote
1answer
264 views

# of iterations in Restricted Boltzmann Machine (RBM)

I have a training set, I provide it (consider a data from training set) to the visible layer. Then the normal process is followed, i.e. Positive Phase-> Negative Phase-> Reconstruction of weights, ...
15
votes
3answers
1k views

Intuition Behind Restricted Boltzmann Machine (RBM)

I went through Geoff Hinton's Neural Networks course on Coursera and also through introduction to restricted boltzmann machines, still I didn't understand the intuition behind RBMs. Why do we need ...
2
votes
1answer
1k views

How is dimensionality reduction achieved in Deep Belief Networks with Restricted Boltzmann Machines?

In neural networks and old classification methods, we usually construct an objective function to achieve dimensionality reduction. But Deep Belief Networks (DBN) with Restricted Boltzmann Machines (...
11
votes
3answers
7k views

How to use RBM for classification?

At the moment I'm playing with Restricted Boltzmann Machines and since I'm at it I would like try to classify handwritten digits with it. The model I created is now a quite fancy generative model but ...
2
votes
1answer
569 views

Pre-train using sigmoid and train using ReLU?

Using RBMs to pre-train a deep net as in this example RBM, the activation function is sigmoid and makes the math much easier. What are the implications after the initial weights are learned using ...
4
votes
3answers
864 views

Training Restricted Boltzmann Machines (RBMs) using gradient descent

Hey I am a little new to the whole RBM entropy/energy training thing, having some trouble understanding and trying to figure whether it is worth the effort needed to understand. Can't RBMs quite ...
1
vote
1answer
58 views

Hidden neuron representation of weights

In an RBM, if we represent the weights learned by the hidden units, they show that the neural net is learning basic shapes. For example, for the mnist dataset, they learn features of the numbers they ...
3
votes
1answer
112 views

Training the parameters of a Restricted Boltzman machine

Why are the parameters of a Restricted Boltzmann machine trained for a fixed number of iterations (epochs) in many papers instead of choosing the ones corresponding to a stationary point of the ...
6
votes
2answers
291 views

How are non-restricted Boltzmann machines trained?

Restricted Boltzmann machines are stochastic neural networks. The neurons form a complete bipartite graph of visible units and hidden units. The "restricted" is exactly the bipartite property: There ...
5
votes
2answers
455 views

What are the advantages of contrastive divergence vs the gradient of the quadratic difference between the original data and the reconstructed data?

In this example I have a RBM with a visible layer and a hidden layer. The original data is "data", the values of the hidden neurons is "hid", the values of the visible neurons calculated from "hid" is ...
2
votes
0answers
165 views

Contrastive divergence in RBM

I have the following code, where x in the input data, w is the weight matrix, bv and bh are the biases for the visible and hidden units. ...
8
votes
1answer
618 views

Why a restricted Boltzman machine (RBM) tends to learn very similar weights?

These are 4 different weight matrices that I got after training a restricted Boltzman machine (RBM) with ~4k visible units and only 96 hidden units/weight vectors. As you can see, weights are ...