# Is it acceptable to use label encoding for nominal categorical data when one hot encoding would create too many features?

I'm working on a short data science project to compare the accuracy of different classification methods. The groups decided to use and compare Random Forest, Naive Bayes and SVM.

The dataset we are using has four categorical features. Each of which has a large number of unique values.

• There are 16537 unique combinations of 17370 unique values in FeaureA.
• There are 13860 unique combinations of 13852 unique values in FeaureB.
• There are 3295 unique combinations of 29 unique values in FeaureC.
• There are 1518 unique combinations of 29 unique values in FeaureD.

From what I've read the RF and NB algorithms should work fine with label encoding but SVM requires one hot encoding. However that would increase the number of features by ~35K. The performance cost seems like it would be significant. Ideally we would use the same encoding for all three algorithms. Would it be better to take the performance hit and try something like PCA for feature reduction?

• Depends on the model. It is fine for RF but will not work for SVM, LR etc. Apr 16 at 17:18
• maybe frequency encoding, be better, towardsdatascience.com/… Apr 17 at 8:29
• How many instances do you have? You would need a very large dataset to avoid overfitting with this level of diversity in the features. Also what do you mean by "X unique combinations of Y unique values"? It looks like each feature is itself a combination of several things, isn't it? Be aware that any feature value which occurs only once in the data is unusable. Apr 17 at 19:05
• The features are things like genre so you have one entry thats action, one thats action;indie, etc Feature A is publisher so nearly all the values are unique I suppose it would be best to eliminate that feature entirely then as it has little to no predictive value? Apr 18 at 13:41

1.) The first thing to do is feature engineering. Try to combine 2 or more features into 1 without losing sense of that variable. For example if you have a dataset of price prediction of used cars, and you have month_of_registration and year_of_registration as 2 of the features. You could combine them to form age variable and then drop the month and year variable. This is the most effective and non invasive way to reduce dimensionality.
2.) Second thing to do is feature selection. This can include any of the 4 techniques namely filter based, wrapper based, embedded and hybrid. This step also includes PCA if you want to use it.