I have a dataset with 5K columns focused on binary classification. I have more than 60 columns. I am trying to find the best features through RFECV approach. Though it produces 30 optimum features, when I plot in graph, I only see 12 features. Please see my code and plot below
model = RandomForestClassifier(n_estimators=100, random_state=0) model_b = LinearSVC(class_weight='balanced',max_iter=1000) # create the RFE model and select 15 attributes rfe = RFECV(model,step=5,cv=5) rfe = rfe.fit(X_train_std, y_train) # summarize the selection of the attributes feat = rfe.support_ fret = rfe.ranking_ features = X.columns print(rfe.n_features_) # this returns 30 as output print(rfe.grid_scores_) this produces the below output
I was expecting to see the grid scores for 30 features and in the plot, I was expecting to see the x-axis to have 30 features as well. But it shows only for 12 features. Similarly if I had only 19 features in my dataset, RFECV returns all 19 as optimal feature which is fine. But again in grid score it only shows 4
q1) Does this mean beyond 12 features, there is no increase in model accuracy?
q2) I assume grid_scores are nothing but weightage/rank which indicates the influence that a feature has on outcome. But how do I get the name of this 12 features?
q3) Wwhy does it show optimum no of features as 30 but grid scores is shown only for 12.
Can you help me with these question please?