I've been learning a little about StyleGans lately and somebody told me that a Multi-Layer Perceptron, MLP, is used in parts of the architecture for transforming noise. When I saw this person's code, it just looked like a normal 8-layer fully connected network (i.e. linear-->relu-->linear-->relu-->...)

Last year, I read Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow 2 by Aurelien Geron and he talks about MLPs. When I read about it, I interpreted his description as that an MLP is not exactly the same as a vanilla fully connected neural network. I didn't fully understand the text and don't have the book anymore so, unfortunately, can't recall exactly what I read so I might have been completely wrong in my understanding of what he wrote.

Is an MLP the same thing as very basic fully connected network?


Yes, a multilayer perceptron is just a collection of interleaved fully connected layers and non-linearities.

The usual non-linearity nowadays is ReLU, but in the past sigmoid and tanh non-linearities were also used.

In the book, the MLP is described this way:

An MLP is composed of one (passthrough) input layer, one or more layers of TLUs, called hidden layers, and one final layer of TLUs called the output layer (see Figure 10-7). The layers close to the input layer are usually called the lower layers, and the ones close to the outputs are usually called the upper layers. Every layer except the output layer includes a bias neuron and is fully connected to the next layer.

("TLU" stands for threshold logic unit)

  • $\begingroup$ Ah great and thanks for including the text from the book so others can see it. I've never seen the term "TLU" used anywhere so that makes it all clear now. $\endgroup$ – zipline86 Mar 25 at 9:46
  • $\begingroup$ The description in the book is certainly not very clear. $\endgroup$ – noe Mar 25 at 9:55
  • $\begingroup$ Maybe that's the terminology that Rumelhart, Hinton, and Williams introduced in the mid-'80s? $\endgroup$ – zipline86 Mar 25 at 14:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.