0
$\begingroup$

I need to implement classical perceptron algorithm from scratch using numpy and pandas for an assignments.

I have done so using this algorithm: enter image description here

I have a linearly seperable dataset of 568 rows and 30 columns. I am training the model on 67% of the data and testing on the remaining 33%.

I ran the algorithm on the raw data and even after the while loop runs 60k times there are ~25 misclassifications in the training data (should be zero).

Then I normalized the data using (x-mean(x))/std(x) [ Replacing each cell value using the mean and std of the column ] and this time the algorithm finished in just ~700 iterations of the while loop! [I got ~92% accuracy on the testing data for this.]

Is there any reason why it finished so quickly on the normalized data and is taking soooo much more time on the raw data?

Improve this question
$\endgroup$
5
  • $\begingroup$ Have you checked the gradient for both cases? I suspect that the gradients in the unnormalized case are quite large, causing quite large weight updates making it difficult for the model to reach the local/global optimum. $\endgroup$
    – Oxbowerce
    Mar 4 at 19:03
  • $\begingroup$ @Oxbowerce I didn't get you.... I don't think I'm using any gradient descent algorithm. I have done a direct implementation of the algorithm in the picture. $\endgroup$
    – Prad
    Mar 4 at 20:15
  • $\begingroup$ You are however multiplying by the value of your inputs in the step where the weights get updated, and a large value for your inputs would mean a large update step. So same logic could be applied with regards to why the model is not converging. $\endgroup$
    – Oxbowerce
    Mar 4 at 21:01
  • $\begingroup$ @Oxbowerce yeah. What do you suggest I do to fix this? $\endgroup$
    – Prad
    Mar 5 at 4:20
  • $\begingroup$ As you already mentioned, normalizing your data is a good one. Additionaly, you could clip the maximum weight update to some value to prevent very large weight updates. $\endgroup$
    – Oxbowerce
    Mar 5 at 10:00

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.