I need to implement classical perceptron algorithm from scratch using numpy and pandas for an assignments.

I have done so using this algorithm: enter image description here

I have a linearly seperable dataset of 568 rows and 30 columns. I am training the model on 67% of the data and testing on the remaining 33%.

I ran the algorithm on the raw data and even after the while loop runs 60k times there are ~25 misclassifications in the training data (should be zero).

Then I normalized the data using (x-mean(x))/std(x) [ Replacing each cell value using the mean and std of the column ] and this time the algorithm finished in just ~700 iterations of the while loop! [I got ~92% accuracy on the testing data for this.]

Is there any reason why it finished so quickly on the normalized data and is taking soooo much more time on the raw data?

  • $\begingroup$ Have you checked the gradient for both cases? I suspect that the gradients in the unnormalized case are quite large, causing quite large weight updates making it difficult for the model to reach the local/global optimum. $\endgroup$
    – Oxbowerce
    Mar 4 at 19:03
  • $\begingroup$ @Oxbowerce I didn't get you.... I don't think I'm using any gradient descent algorithm. I have done a direct implementation of the algorithm in the picture. $\endgroup$
    – Prad
    Mar 4 at 20:15
  • $\begingroup$ You are however multiplying by the value of your inputs in the step where the weights get updated, and a large value for your inputs would mean a large update step. So same logic could be applied with regards to why the model is not converging. $\endgroup$
    – Oxbowerce
    Mar 4 at 21:01
  • $\begingroup$ @Oxbowerce yeah. What do you suggest I do to fix this? $\endgroup$
    – Prad
    Mar 5 at 4:20
  • $\begingroup$ As you already mentioned, normalizing your data is a good one. Additionaly, you could clip the maximum weight update to some value to prevent very large weight updates. $\endgroup$
    – Oxbowerce
    Mar 5 at 10:00


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.