I need help with which features to use for clustering

Question

I am using this dataset: https://www.kaggle.com/datasets/sobhanmoosavi/us-accidents

and so far I have successfully cleaned the dataset as well as reduced the size of the features and records.

I have some numeric features that I have standardized and there are some categorical features remaining that I have not touched. Now I need to perform clustering with Kmeans/Dbscan etc and compare their results, I read that since there are a lot of features (38) I can perform PCA on the numeric features and use the PCA components for clustering.

I also read that using PCA on categorical features is not very correct although it may produce a result. The way I want to do clustering is to remove the target feature('Severity') and do unsupervised learning on the remaining features to see how well it does to predicting the target feature.

Now for the question: Since I have so many features, even if I do PCA on the numeric ones and use the PCA components for clustering, will the result be okay? And what to do with categorical features? One hot encode them and put them in the PCA or don't use them for clustering at all?

All the examples I found used a small dataset with less than 10 features for clustering.

Thank you!

Answer 1

PCA is interesting but hard to interpret because it is a linear algorithm.

Consequently, the result using many features will be probably not OK, overall if you have non-linear correlations or complex variances.

I would recommend keeping PCA to do some studies on 3 features and using non-linear dimensional reduction algorithms such as t-SNE, or UMAP.

As a picture is always better than thousands of words, here you can test the difference between PCA, t-SNE, and UMAP with different datasets: https://projector.tensorflow.org/