2
$\begingroup$

I am trying to understand which model might work for a given problem before trying the models, I find this case against my knowledge. Please guide what I am missing. I am new to Data Science.

Here is the graph which I got through PCA : enter image description here

Now you can see the boundaries are very much overlapping. The theory for SVM says that this model might work best with overlapping non linear data, which does not seems to be this case.

But still its able to identify all data in test set. So can you provide some clarity on why SVM performing good in this.

So my final results it is below order:

  • Logistic Regression and SVM are same (Accuracy Score : 1.0)
  • Random Forest (Accuracy Score : 0.9680851063829787)
  • KNN (Accuracy Score : 0.925531914893617)

other details :

  • feature set : 40
  • sample data : around 500
$\endgroup$
1
$\begingroup$

I assume you applied SVM to your initial data and use PCA only for visualization. I this case:

I guess your projection via the PCA is not showing the real picture.

You should check first how much of your data is explained with the first two principal components of the PCA. Your projection might change to much the structure of your data so that its not seperable anymore. In case the projection to the selected principal components is not changing your data too much, it could be that you maintain the seperability.

Finally, note: If the projection is linearly seperable, then so is your data. If the projection is not linearly seperable, you cannot conclude about seperability of your data.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.