4
$\begingroup$

When we talk about Perceptrons, we say that they are limited for approximating functions that are linearly separable, while Neural Networks that use non-linear transformations are not.

I am having trouble understanding this idea of linear separability. Specifically, does it apply only for binary classification or it generalizes for N classes? If so, how does a linear decision boundary for lets say 4 classes even look like?

$\endgroup$
-1
$\begingroup$

I will try to explain this problem in a classification scenario. Say we begin with two classes of images say dog and cat. And we are asked to classify the images whether the given test image is dog or cat. Imagine the images that we are given is $8 \times 8$. So each image is represented by an $8 \times 8=64$ dimensional vector. For the training dataset, one such training image will form a data point in this 64-dimensional space. After lots of training data of cat and dog images say 1000 training cat images and 1000 training dog images, we get 2000 such points in the 64-dimensional space. If in that space, the points can be classified using a 64-dimensional hyperplane, then it's linearly separable in an easy way. If not, we can use kernel which is a nonlinear function of the given image, which will transfer the 64-dimensional space into a higher space (more than 64, where we can find linear separability. In a word, each problem is linearly separable. For more information, you can go through Linear Support Vector Machine and Kernel Support Vector Machines.

$\endgroup$
  • $\begingroup$ Hi, thanks for spending time to answer my question! Unfortunately it is NOT the answer I was looking for. First of all, you did not address my question. Is linear separability property that applies only to binary classification or can it be generalized to N classes? Second, I think that you are wrong about hyperplane in your example. Dimensionality of a hyperplane is by definition 1 less then the ambient space. So in your example, hyperplane that represents decision boundary should be 63 dimensions, not 64. $\endgroup$ – Stefan Radonjic Jan 24 at 15:41
  • $\begingroup$ Answer to the first question: Yes. It can be generalized to N classes. $\endgroup$ – Debasish Jana Jan 28 at 15:45
  • $\begingroup$ Second question: You are absolutely right that the hyperplane will be of 63 dimensions. $\endgroup$ – Debasish Jana Jan 28 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.