1
$\begingroup$

I have a data set with huge number of features ( Approximately 3000) and a binary target variable . The reason I have too many features is because of one hot encoding many categorical variables in my data set .

I think logistic regression might only work with small number of features .

So , given that I have many features , which algorithm should I use for better classification score ?

My aim is to increase the ROC-AUC metric for this classification task .

Is it better to use SVM or Neural networks ?

Improve this question
$\endgroup$
2
  • $\begingroup$ How many rows of data do you have? $\endgroup$
    – Neil Slater
    Commented Sep 13, 2020 at 10:17
  • $\begingroup$ 8844 rows of data $\endgroup$
    – Bharathi A
    Commented Sep 13, 2020 at 11:25

1 Answer 1

Reset to default
1
$\begingroup$

First thing that comes to my mind is to do different encodings. There are some ways to deal with high cardinality categorical data such as: Label Encoding or the famous target encoding. Before anything else I will recommend changing the encoding type.

But, since your question about which predictor use with small and space data. I will go still with logistic regression, decision tree or SVM. When data is small all algorithms tend to work quite similar.

Things like Random Forest might perform well since they do bootstrapping what tends to be a way to sample your data with replacement.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks for the answer . But , I have studied that Label encoding might cause an implicit ordering of categories which can mislead the model . So , will it affect my classification ? $\endgroup$
    – Bharathi A
    Commented Sep 14, 2020 at 14:53
  • $\begingroup$ Yes, that is the problem of label, consider then target encoding. In this way your categories will be ordered with a correlation to the target distribution. Be aware of overfitting with this encoding technique $\endgroup$
    – Carlos Mougan
    Commented Sep 14, 2020 at 15:30
  • $\begingroup$ in the category encoder library there is a lot of encoders. You might find some useful contrib.scikit-learn.org/category_encoders $\endgroup$
    – Carlos Mougan
    Commented Sep 15, 2020 at 6:11
  • $\begingroup$ Okay , I will check them out , thanks a lot. $\endgroup$
    – Bharathi A
    Commented Sep 15, 2020 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.