1
$\begingroup$

I am working on the Webspam UK 2006 dataset.I have a dataset with around 98 different features. I am planning to apply Decision Tree C4.5 classifier on it. I have done feature selection using random forest and I am able to get the MeanDecreaseGini for each feature. Now my question is out of these 98 features how do I decide the total number of features to drop?

> importance(model_rf)
       MeanDecreaseGini
hostid        16.470372
HST_1          8.243852
HST_2          7.269966
HST_3          6.867545
HST_4         12.726805
HST_5         11.496738
HST_6         16.547676
HST_7         11.990072
HST_8         12.133017
HST_9         13.215381
HST_10        15.987938
HST_11         7.452778
HST_12         6.426637
HST_13         6.989566
HST_14         7.523594
HST_15         9.180248
HST_16        17.578608
HST_17        13.813119
HST_18        12.469869
HST_19         8.205692
HST_20         9.928898
HST_21         8.204766
HST_22        10.453883
HST_23         7.250751
HST_24         7.768237
HMG_25         6.888539
HMG_26         5.202057
HMG_27         7.272760
HMG_28         9.496095
HMG_29        22.814004
HMG_30         8.385849
HMG_31         8.962740
HMG_32         8.788728
HMG_33         8.500586
HMG_34         9.605438
HMG_35         6.916501
HMG_36         5.499630
HMG_37         7.707432
HMG_38         7.170538
HMG_39         7.519015
HMG_40        20.864126
HMG_41        13.891407
HMG_42        17.097263
HMG_43         5.907863
HMG_44         8.961656
HMG_45         6.505953
HMG_46         8.198271
HMG_47         6.683629
HMG_48         7.521221
AVG_49         6.978512
AVG_50        10.625453
AVG_51        11.787007
AVG_52        10.501896
AVG_53         8.998260
AVG_54         7.169787
AVG_55        19.950379
AVG_56        10.941224
AVG_57        11.911999
AVG_58        11.145681
AVG_59        23.529763
AVG_60         8.726543
AVG_61        11.881046
AVG_62         6.889028
AVG_63        12.834768
AVG_64         6.642953
AVG_65        16.349392
AVG_66         6.885495
AVG_67        15.492990
AVG_68         7.982041
AVG_69         7.274046
AVG_70         6.883014
AVG_71         6.684649
AVG_72         7.105827
STD_73         6.440758
STD_74         6.952049
STD_75         7.192637
STD_76         8.917629
STD_77        23.578747
STD_78         9.402747
STD_79        39.137215
STD_80        11.908751
STD_81        34.713739
STD_82         9.006674
STD_83        42.645293
STD_84         9.555861
STD_85        13.003373
STD_86         6.609194
STD_87        13.641984
STD_88         6.627306
STD_89         8.565469
STD_90         6.497847
STD_91         8.985288
STD_92         6.702219
STD_93         8.187475
STD_94        10.954584
STD_95         8.111179
STD_96         8.080897
$\endgroup$