I was trying to use feature importances from Random Forests to perform some empirical feature selection for a regression problem where all the features are categorical and a lot of them have many levels (on the order of 100-1000). Given that one-hot encoding creates a dummy variable for each level, the feature importances are for each level and not each feature (column). What is a good way to aggregate these feature importances?
I thought about summing or getting the average importance for all levels of a feature (probably the former will be biased towards those features with more levels). Are there any references on this issue?
What else can one do to decrease the number of features? I am aware of group lasso, could not find anything easy to use for scikit-learn.