2
$\begingroup$

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is there a general rule ?

$\endgroup$
3
$\begingroup$

Yes, the perceptron learning algorithm is a linear classifier. If your data is separable by a hyperplane, then the perceptron will always converge. It will never converge if the data is not linearly separable.

In practice, the perceptron learning algorithm can be used on data that is not linearly separable, but some extra parameter must be defined in order to determine under what conditions the algorithm should stop 'trying' to fit the data.

For example, you could set a maximum number of iterations (or epochs) for the algorithm to run, or you could set a threshold for the maximum number of allowed misclassifications.

$\endgroup$
3
  • $\begingroup$ The extra parameter you are talking about is the adaptive learning rate? $\endgroup$ Oct 17 '17 at 18:03
  • 1
    $\begingroup$ Not quite. I'm not an expert, but I believe different parameters could be chosen. The simplest would be to just set an upper limit on the number of possible iterations. An adaptive learning rate just means your learning rate gets smaller over time. If your classes are not linearly separable changing the learning rate wouldn't help the algorithm converge. $\endgroup$
    – Ben
    Oct 17 '17 at 19:38
  • $\begingroup$ Here's a good reference: sebastianraschka.com/Articles/2015_singlelayer_neurons.html $\endgroup$
    – Ben
    Oct 17 '17 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.