2
$\begingroup$

I am facing some confusion regarding the terminologies assocaiated to classification and regression problems esp. using the MLP and Perceptron models. These are the following:

1) When the data is linearly inseparable, we use MLP. Here what is meant b "data"--is it the response or the input feature that is linearly inseparable?

2) If it is linearly inseparable then does it mean that the mapping function from input to output will always be non-linear? Hence, we prefer MLP or the latest new models such as deep learning?

3) Linear regression fails in the case of linearly inseparable data or can linear regression work for inseparable data but if the function mapping is nonlinear then it fails?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

When the data is linearly inseparable, we use MLP. Here what is meant by "data"--is it the response or the input feature that is linearly inseparable?

This means that a linear function of the input features is unable to separate the response.

To answer your question a bit more directly: Given only a linear function of the inputs, the response is the thing that's inseparable.

If it is linearly inseparable then does it mean that the mapping function from input to output will always be non-linear? Hence, we prefer MLP or the latest new models such as deep learning?

Yes. If the mapping from input to output were linear, then the output would necessarily be linearly separable by the input.

Linear regression fails in the case of linearly inseparable data or can linear regression work for inseparable data but if the function mapping is nonlinear then it fails?

Linear regression will never be able to perfectly separate linearly inseparable data. Consider the following example, where the input features are x1 and x2, and the output is the color:

linearly-inseparable data

It doesn't matter how you draw a line in the 2D space - you'll never be able to separate the colors. The same idea applies in higher dimensions.

I hope that helps!

Improve this answer
$\endgroup$
7
  • 1
    $\begingroup$ Thank you so much for the answers. One last thing, can you please confirm -- is MLP a nonlinear model since it use multiple nonlinear functions to separate the data or to map the input to output? $\endgroup$
    – Sm1
    Oct 8, 2019 at 18:33
  • $\begingroup$ That's right - MLP can capture nonlinear mappings from input to output, as long as the activation function is nonlinear. $\endgroup$
    – zachdj
    Oct 8, 2019 at 19:01
  • $\begingroup$ Also, this is a bit pedantic, but I'm not sure if it's correct to say that MLP "is nonlinear because it uses multiple nonlinear functions to map input to output". Rather I would say that an MLP is a nonlinear function from input to output. $\endgroup$
    – zachdj
    Oct 8, 2019 at 19:05
  • $\begingroup$ Now confused -- mathworks.com/help/deeplearning/ref/fitnet.html is the tutorial that I am following.(1) If number of layers = 1 with 10 hidden neurons (as in second figure) then it is essentially a neural network and not a MLP. Is that correct? (2) As per your last comment, did you mean the activation function of the last layer determines nonlinear/linear? In this picture, the last layer is the output layer and the activation function looks like an identity/linear. But the hidden layer has sigmoid activation function which is nonlinear. Therefore, is this MLP a nonlinear function? $\endgroup$
    – Sm1
    Oct 8, 2019 at 20:31
  • 1
    $\begingroup$ Sorry for the confusion. (1) All MLPs are neural networks, in the same way that all squares are rectangles. MLPs are a subclass of neural network that are fully connected, feedforward with sigmoid activation in the hidden layer(s). (2) I meant that the activation of the hidden layer was nonlinear. The activation function of the last layer will depend on the problem. $\endgroup$
    – zachdj
    Oct 9, 2019 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.