Is a multi-layer perceptron exactly the same as a simple fully connected neural network?

Question

I've been learning a little about StyleGans lately and somebody told me that a Multi-Layer Perceptron, MLP, is used in parts of the architecture for transforming noise. When I saw this person's code, it just looked like a normal 8-layer fully connected network (i.e. linear-->relu-->linear-->relu-->...)

Last year, I read Hands-on Machine Learning with Scikit-Learn, Keras, and TensorFlow 2 by Aurelien Geron and he talks about MLPs. When I read about it, I interpreted his description as that an MLP is not exactly the same as a vanilla fully connected neural network. I didn't fully understand the text and don't have the book anymore so, unfortunately, can't recall exactly what I read so I might have been completely wrong in my understanding of what he wrote.

Is an MLP the same thing as very basic fully connected network?

Answer 1

Yes, a multilayer perceptron is just a collection of interleaved fully connected layers and non-linearities.

The usual non-linearity nowadays is ReLU, but in the past sigmoid and tanh non-linearities were also used.

In the book, the MLP is described this way:

An MLP is composed of one (passthrough) input layer, one or more layers of TLUs, called hidden layers, and one final layer of TLUs called the output layer (see Figure 10-7). The layers close to the input layer are usually called the lower layers, and the ones close to the outputs are usually called the upper layers. Every layer except the output layer includes a bias neuron and is fully connected to the next layer.

("TLU" stands for threshold logic unit)