1
$\begingroup$

I have a dataset which contains : 94 numeric features + 56 categorical features

I am trying to build a classifier to predict Target (disease/healthy). 2. Rows : 1812 3. Class imbalance ( Majority class(healthy : 1200))

Strategies i tried :

  1. I tried SMOTE on training data( 80/20 split).
  2. As number of features are 150 . I wanted to use feature selection . Filter methods (fs) dont work so well on mixed. So i used Wrapper fs algorithm( recursive feature elimination) on the full feature set. Used random forest for rfe().
  3. I got a subset of 23 features from rfe (all numeric) . Wonder if rfe works with factor variables? or should i use dummy encoding first and then rfe?
  4. I even tried hybrid feature selection alg. like Genetic algorithm but sensitivity is very low after randomforest (50 % for disease class)
  5. I am trying to achieve a high sensitivity . I am concentrating on random forest as of now as my base classifier as it works well with mixed datatype.

Issues: my best Cross validated accuracy is 70. Sensitivty : 50 . How can i improve sensitivity for disease class?

Feature selection for mixed data ? I also tried PCA and FAMD for dimensionality reduction, but PC1 explains only 8 % variance. So thats out of the way.

Thank you for your time

Improve this question
$\endgroup$
6
  • $\begingroup$ Random forest works by building many decision trees using CART. The CART trees will internally convert the continuous variables to factors. CART (and therefore RF) loves factors. $\endgroup$
    – G5W
    Commented Jul 19, 2019 at 19:40
  • $\begingroup$ It's not a very big class imbalance. What happens if you simply remove half of the majority class instances? (it's likely to increase the recall and decrease the performance overall). Also there's a chance that you're removing too many features, did you try keeping more of them, or even all of them? $\endgroup$
    – Erwan
    Commented Jul 20, 2019 at 1:25
  • $\begingroup$ @Erwan Thanks for your reply. I trained rf on complete set of data performance is worse than using the 23 numeric features i got from rfe. I even tried tomek with smote it improved sensitivity marginally. I tried undersampling majority to make almost equal to minority , it gives lower senstivity. Any good feature selection techniques for mixed data that you can suggest? rfe: done , GA: done THanks $\endgroup$
    – LukeSkywalker
    Commented Jul 20, 2019 at 1:44
  • $\begingroup$ Genetic selection is the best option I can think of for feature selection since it will try many subsets to find an optimal one. You already did it but maybe you can try to change the parameters of the genetic learning in order to give it a better chance to reach the optimal set of features? (more generations, larger generations, etc.) Unfortunately it's also possible that you can't do better than this with this data. $\endgroup$
    – Erwan
    Commented Jul 20, 2019 at 1:53
  • $\begingroup$ @Erwan. Can you suggest an implementation of GA in r. i tried the one in R caret and also by github.com/pablo14/genetic-algorithm-feature-selection. Not sure about the internal implementation is correct or not for the github though. Also i input categorical features as factors to GA . Do you think 1 hot encoded factors should be rather input to GA? $\endgroup$
    – LukeSkywalker
    Commented Jul 20, 2019 at 2:12

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.