I'm working on a short data science project to compare the accuracy of different classification methods. The groups decided to use and compare Random Forest, Naive Bayes and SVM.
The dataset we are using has four categorical features. Each of which has a large number of unique values.
- There are 16537 unique combinations of 17370 unique values in FeaureA.
- There are 13860 unique combinations of 13852 unique values in FeaureB.
- There are 3295 unique combinations of 29 unique values in FeaureC.
- There are 1518 unique combinations of 29 unique values in FeaureD.
From what I've read the RF and NB algorithms should work fine with label encoding but SVM requires one hot encoding. However that would increase the number of features by ~35K. The performance cost seems like it would be significant. Ideally we would use the same encoding for all three algorithms. Would it be better to take the performance hit and try something like PCA for feature reduction?