To my understanding, a learning method is used to train a neural network on a dataset by training it's outputs.

I'm aware that the same neural network can be run backwards to generate data.

I'd like to learn how to reverse a neural network's algorithm to generate.

Any ideas?

See this video.

  • $\begingroup$ Could you add a little about your level of understanding of neural networks. For example, if an answer used terms such as weights, transfer functions, hidden layers etc, would you understand it? How about the equations for feed-forward and backpropagation training? Have you heard about Restricted Boltzmann Machines (RBMs)? $\endgroup$ Mar 30, 2016 at 12:47
  • $\begingroup$ I have a rectangular neural network that is feed forward. Going one way the inputs are presented with training data, and the specific outputs are trained. The method of learning is by genetic algorithm, not backpropagation. I wish to reverse the algorithm wherein the outputs become the inputs and the inputs become the outputs, the outputs (formerly the inputs) generating a set of training data that it "believes in" (see the video) for a certain fixed input (formerly the output). How would one make this happen? $\endgroup$
    – Sam
    Mar 31, 2016 at 22:30
  • $\begingroup$ The demo from the video is probably a RBM. en.wikipedia.org/wiki/Restricted_Boltzmann_machine You will need to change your learning algorithm in that case (or just use a library that supports RBMs). Also see class.coursera.org/neuralnets-2012-001 - although this doesn't cover code for a generative network in detail, it does describe the maths. $\endgroup$ Apr 1, 2016 at 7:53

1 Answer 1


Feed-forward neural networks are discriminative models, i.e. they model $P(y|x)$.

Something like restricted Boltzmann machines look a lot like neural networks, and share some of the proprieties, and are generative models, i.e. they model $P(y,x)$ --- which, of course, you can always turn into a $P(y|x)$ using Bayes' rule, but the opposite is not true.

Libraries like Theano have support for RBMs.

EDIT: since the writing of my reply, generative adversarial networks have been invented. (Actually, they were invented in 2014, but only in 2016 were they taken to their full potential.) In these networks, you feed noise to the neural network and it generates data samples. This is achieved by co-training it against a regular discriminative neural network. Both networks are trained at the same time: the discriminative one must classify each sample as being either true or synthetically created by the other neural network. And the generative neural network must fool this one, so they get better and better, and, in the end, you have both a discriminative and a generative model!


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.