New answers tagged categorical-data
0
There are multiple ways you can do this some of which are:-
1.) Use L2 Regularization to determine which out of the 5 features are contributing
more in predicting the target.
2.) Use Tree based and gradient boosting based model to calculate the feature importance of each features. Here is an article that uses various models to calculate the feature ...
0
One option is one-hot encode the categorical features. Then increment the value of the feature to act as a counter for the number occurrences.
For example - if you are modeling a shopping cart, one feature would be "apples" and there would a count for the number of apples in the cart.
1
With only three observations, a model won't be able to make any confident estimate for the contribution. I think your best bet is to use some domain knowledge to encode the column (no=0, unknown=1, yes=1.2? 0.7? ...), or just drop the column if it's not informative. Dropping the rows is fine too, but in production what would you want the model to do if it ...
1
How you handle this depends on the variable's importance in predicting the target variable. A quick test would be to compute the event-rate of target variable across the distinct classes and determine if the distinct classes significantly separate the target variable. If there is no separation, dropping the variable makes more sense, otherwise you consider ...
1
I second @Erwan's observation that which categories to censor depends on the purpose of the analysis. This dataset is bank credit data collected with a view to a single outcome variable:
21 - y - has the client subscribed a term deposit? (binary: 'yes','no')
I feel comfortable in discarding the 3 out of 40K observations of "yes" for this purpose, ...
6
It totally depends on the target task and the importance of the "tiny class" for this task:
In some tasks with multiple classes where there is no particular emphasis on any specific class, "tiny classes" can simply be discarded.
But in the case of this dataset, the natural target task is to detect default cases, and there's little point ...
0
I'm not very knowledgeable about this but here a some ideas:
Statistical testing: in order to know whether there is any statistical difference between the values in two categories, one can use the Student t-test (for normal distributions) or the Wilcoxon test (for any distribution).
In the case where there is a significant difference there are methods to ...
Top 50 recent answers are included