How do I produce generative neural networks?

To my understanding, a learning method is used to train a neural network on a dataset by training it's outputs.

I'm aware that the same neural network can be run backwards to generate data.

I'd like to learn how to reverse a neural network's algorithm to generate.

Any ideas?

See this video.

• Could you add a little about your level of understanding of neural networks. For example, if an answer used terms such as weights, transfer functions, hidden layers etc, would you understand it? How about the equations for feed-forward and backpropagation training? Have you heard about Restricted Boltzmann Machines (RBMs)? Mar 30, 2016 at 12:47
• I have a rectangular neural network that is feed forward. Going one way the inputs are presented with training data, and the specific outputs are trained. The method of learning is by genetic algorithm, not backpropagation. I wish to reverse the algorithm wherein the outputs become the inputs and the inputs become the outputs, the outputs (formerly the inputs) generating a set of training data that it "believes in" (see the video) for a certain fixed input (formerly the output). How would one make this happen?
– Sam
Mar 31, 2016 at 22:30
• The demo from the video is probably a RBM. en.wikipedia.org/wiki/Restricted_Boltzmann_machine You will need to change your learning algorithm in that case (or just use a library that supports RBMs). Also see class.coursera.org/neuralnets-2012-001 - although this doesn't cover code for a generative network in detail, it does describe the maths. Apr 1, 2016 at 7:53

Feed-forward neural networks are discriminative models, i.e. they model $P(y|x)$.
Something like restricted Boltzmann machines look a lot like neural networks, and share some of the proprieties, and are generative models, i.e. they model $P(y,x)$ --- which, of course, you can always turn into a $P(y|x)$ using Bayes' rule, but the opposite is not true.