2
$\begingroup$

I have a dataset that I will be using to build a classifier on. Below I have plotted the First and the Second Principal Component of the data using sklearn.preprocessing.PCA. Since the two different classes are not well separated a linear classifier will not work here.

My question is which classifier would be best for this scenario.

My research brought be to KNN. But My intuition says the class ratio is highly imbalanced a large value of k in KNN would always tend towards the larger class count. It will be a nightmare to train it on SVM since therw are to many observations in the dataset and it will take too long.

PCA with n_components = 2

$\endgroup$

2 Answers 2

2
$\begingroup$

Note that doing a dimensionality reduction with the target can lead you to the manifold problem. You can see in the image. What ends up happening is that the target information is lost.

Manifold problem with swiss roll

The information that you provide is not enough to make a guess of what algorithm will be better.

Normally reducing the dimensionality of the problem to a lower dimension space in order to plot where you want to have the decision boundaries is not a good idea.

It's really hard to have a human understanding of how do these algorithms do the decision boundaries in a large scale dimension, that is why it is just better to have an empirical approach. Try a few of them, select a metric, and choose the one that has a higher score.

$\endgroup$
0
$\begingroup$

The information you are providing here is really not sufficient. The PCA image is only showing that using the first two principal components will not bring any benefit. Still, it could be that using the first three (or more) principal components will help.

If you really want some help, you need to provide more information. What is the structure and distribution of your input data?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.