Queries regarding feature importance for categorical features

Question

Queries regarding feature importance for categorical features:

Context: I have almost 185 categorical features and these categorical features have either 2 or 3 or 8 or 1 or sometimes 4 categories, null's also. I need to select top 60 features for my model. I also understand that features needs to be selected based on business importance OR feature importance by random forest / decision tree.

Queries:

1. I have plotted histograms for each feature (value count vs category) to analyse. What is the approach to select whether feature is important?

2. What is the standard practice followed across data science industry to get the feature importance from categorical data?

3. Is there basic and elegant way to select the top important features?

4. How do I engineer these categorical features?

Answer 1

From what I understand, you have 185 features and out of them you want to choose the 80 most informative/ important?

It really seems like the main focus of all algorithms for decision trees, which is how to decide which attribute is the best for a split? I suggest you do some reading in the context of decision trees. For example: DECISION TREES - Presentation

What you are looking for is an Impurity-Measure which can help determine the Goodness-of-Split due to some discrete attribute (basically measuring information gain):

There are 3 measures which are used most commonly - Gini index, miss-classification error and entropy.